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ABSTRACT 

The data in this suaaary portray full-tiae, 
part-time, and total headcount enrollment for each caapus and the 
entire university of Haine for the years from 1970 to 1974. Full^tiae 
equivalent (FTE) enrollment is also shovn. For fall 1974, tvo 
different FTE enrollments are shovn. The first is comparable to the 
previous years, the second FTE figure reflects the nev method for 
equating part-time students to full-time students on the basis of 
student credit hours generated* .Both FTE figures are shovn so that 
the difference created by a nev methodology may be noted. From fall 
1973 to fall 1974, ^headcount enrollment increased by 2 percent. 
Part-time enrollment increased by 1 percent and full-time enrollment 
by 1.5 percent. FTE enrollment, using a similar method both years, 
increased by 2.4 percent. Regular day-time enrollment increased by 
5.3 percent (headcount) while continuing education enrollment 
declined by 5 percent. (Author/PG) 
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THE OFFICE OF FINANCIAL PLANNING OF THE UNIVERSITY OF 
MAINE PUBLISHED INFORMATION SHEET #9, JUNE 13, 1973 
SUHHARIZINC DEGREE CBSDIS ENROLLMENT AT EACH CAI-IPUS 
FOR EACH XEAR FROM FALL 1965 THROUGH FALL 1972. 



INSTITUTIONAL RESEARCH ME^30 #?, JANUARY 2% 1974, 
PROVIDED SD-OLAR DATA FOR THE 5-YEAR PERIOD FROM 
FALL 1969 - FALL 1973. 



THIS MEI'JO SHOWS DATA FOR THE 5-YEAR PERIOD FROM 
FALL 1970 - FALL 1974. A MORE DETAILED REPORT AND 
ANALYSIS WHi BE FORTHCOMING. 
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The data in this summaxy portray fuUrtimei parb«-tijne and total 
headcount enrollment for each campus and the entire University of Maine 
for the years from 197 ^ to 1974* PuH-time equivalent (FTE) enroUinent 
is also shown* 

For Fall 1974f two different FTE enrollments are shown* The first, - 
which is comparable to the previoiis years, is computed as follows: 

1* jthdergraduate students 

1 ftiLl-time student » 1 FTE student 

1 part~time student , « FTE student 

2* Graduate students 

regular day students « 1 FTE studexxt 

CED students are equated in the same manner as 
undergraduate students 

The second FTE figure which is in parenthesis reflects a new method 
for equating part^ime students to full-time students on the basis of student 
credit hours generated, as follows: 

1* Undergraduate level programs: 

Total student credit hours generated ^ ^ sttadent 
15 

2* Graduate level programs: 

Total student credit hours generated , ^ ^ ^ 
' g ' ' ■ ■ » 1 FTE student 

Both FTE figures are being shown so that the difference created by a 
new methodology may be noted* 



(over) 



From Fall 1973 to Fall 1974t headcount enrollment increased by 0$. 
Part-^ime enrollment increased by IjJ and full-time enrollment by 
FT£ enrollment^ using a similar method both years^ increased by 2«4^« 

Regular day-time enrollment increased by 5#3^ (headcount) while 
Continuing Education enrollment declined by 



The changes vaxy f rom campus to campus* 
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Institutional Research Memorandum No* 14 will contain more detailed 
information and analysis of enrolbnent data« 



Distribution: 

Chancellor & Staff 

Vice Chancellors & Staffs 

Presidents 

Board of Trustees 



flQ. a FORMULATION OF ACTION SKILLS IN A CURRICULUM* 



Action tkilit 

At the time of graduation from medical school it is unrealistic to 
expect all students to havo acquired a level of professional competence 
sufficient to justify Mipendent management of any significant numt)or 
of heaitt) problems. There are some areas, however, in which this 
competence should have been achieved, and these are listed belcw. 
In addition, any student should be able to formulate a general plan of 
management for the disease processes he is expected to recognize 
(and which are listed in an earlier section of this document). He should 
also be prepared to outline a diagnostfa approach to prominent 
presenting symptoms and signs not immediately recognizable a& specific 
diseases. 

I* Indapeiideiil inaiiagenient 

— Routine screening procedure for adults, adolescents, children, 

infants 

— Pre-marital and marital counselling, including contraceptiva advice 
— Normal pregnancy and delh^ry 

— Emergency care, acute traumatic injury 
— Emergency care, cardiac arrest 

— Emergency care, acute poisoning (aspirin, lye, barbiturate, hydro- 

cart)ons, mescaline) 
— Anterior nasal haemorrhage 

— etc 

2. Qeneral wumjeinant plan 

— See list of diseases to be recognized 

— Physical rehabilitation for patients who have suffered myocardial 

infarction, stroke 
— Schedule of immunization for intents, special risk groups 
— To minimize health hazards according to age, sex, race, socio- 
economic status and place of residence 

— etc. 

3. Dtognoetle plan -tymplome and eigne 

— Fever 

— Pallor 

— Weakness 
— Jaundice 
— Vomiting 

— etc. 



« Extracted from the curriculum of the Abraham Lincoln School of Medfclna, Unlvarslty 
of nilnols Collage of Madlclna. 
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alike can be freed from the artifidal constraints of instractional time 
and sequence* 



CURRICULUM PLANS 

Traditionally a curriculum plan takes the form of a descr^ve outline 
of courses, their arrangement and sequence, the time assigned to them, 
the omtent that win be covered in them, and the instructional methods 
that win be employed. Such a document guides faculQr in prq)aring 
their instruction, tdls students where to go and what e^q)erieDces they 
can expect, and i^vides for external reviewers the kind of evidence 
that is generally sou£|it when judgements are made about educational 
quality. But it tells none of tiiese individuals or groups vfbat the 
product of the curriculum is intended to be. 

The "accountable curriculum"" plan, on the other hand, would also 
be an outline but would focus upon a description of learning outcomes 
and the varied resources to which students might turn to achieve those 
goals. These resources migiht include formal course offerings, where 
they were appropriate and useful, but only as one of many modes that 
mi^t be employed Hiis document would also serve faculty, but as a 
constant reminder that there is no sin^-cuHiciifom-optimal-lei^>all- 
students and as regular encouragement to create options and alternatives 
rather than pursue the unreachable goal of the curriculum. It would 
serve students as a definitive guide to what they are expected to 
become, not merely a timetable of what they are expected to do. And 
it would serve external reviewers (as weU as internal faculty and 
administration) as a point of reference for jud^g educational quality 
in terms of planned learning outcome rather than designed instructional 
content 

With such a curriculum plan in band the task of evalqating student 
achievement is substantially simplified, since the first and most critical 
step in that process — spedfication of learning objectives — will have 
been completed as part of the planning. And this is where evaluation 
must begin, rather than being tacked on at the end. 

But equally important is the opportunity such a plan provides for 
examining the outcome in terms of the social, economic, and political 
as well as the academic determinants. In any careful programme review 
each such determinant in turn should be used as a criterion against 
which the curriculum is judged. To the extent that it fails to meet these 
criteria, both the plan and the planning process must be re-examined 
to determine what went wrong. Were the determinants appropriately 
identified? Were the necessary interests and expertise represented in 
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the planning group? Were priorities clearly established and did the 
planners adhere to these priorities in making curriculum decisions? 
Were those priority decisions reflected in the final curriculum plan as 
well as in the instructional tactics? It is through such systematic review 
that faults are identified and corrected 

But even if the criteria seem to have been met» the task of review 
does not end; instead the cycle should begin once more. Academic, 
social, economic, and political detenmnants are not static; the world 
changes, and with it the curriculum must also change to meet new 
needs as well as to exploit new opportunities. Curriculum theory 
demands that the process of plaiming be continuous, dynamic, and 
never-ending if it is to serve best the needs of individual students, 
educational institutions, and the society to which both are ultimately 
accountable. The time is at hand for that theory to be more fully 
reflected in contemporary curriculum planning practices. 



ADDmONAL READING 

Bcnnis, W. G. ct al (1969) The planning of change, New York, Holt, Rinehart 
& Winston 

Berman, L. M. (1968) New prioriiies in the curriculum, Columbus, Ohio, Merrill 
Bruner, J. S. (I960) The process of education, Cambridge, Mass., Harvard 
University Press 

Goodman, P. (1964) Compulsory mis-education. New York, Horizon Press > ^ 
Mager, R. F. (1962) Preparing instructional objectives, Palo Alto, Calil, Fearon 
Schwab, J. J. (1964) Structure of the disciplines: meanings and significance. 

In: Ford, G. W. & Pugno, L., ed.. The structure of knowledge and the 

curriculum, Chicago, Rand McNaUy 
Tyler, R. W. (1949) Basic principles of curriculum and instruction, Chicago, 

University of Chicago Press 
Tyler, R. W. et al. (1967) Perspectives of curriculum evaluation. No. 1, Chicago, 

Rand McNaUy 
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SIMULATION IN INSTRUCTION 
AND EVALUATION IN MEDICINE 



Christine £L McGiure^ & Frederick H. Wezeman* 



* Sound dinical judgement requires not only a broad knowledge and 
understanding of health and disease and relevant professional habits and 
attitudes concerning the delivery of health care but also the application 
of complex cognitive and psychomotor skills. These skiUs include the 
collection^ processing, and interpretation of data; using a variety of 
resources (including expert advice); ordering priorities and decision- 
making; manipulating a situation to alter it Oncluding interacting with 
others to effect change); and monitoring the consequences of these 
manipulations and modifying decisions or actions to respond to the 
altered situation. Conventional methods of instructing and evaluating 
students^ residents^ and clinicians are often not optimally suited to the 
development and assessment of these important components of physician 
competence. Analogous problems in otfier fields have sometimes been 
resolved by the introduction of sophisticated simulators designed for 
both teaching and testing. 

For example^ with the advent of the space age, scientists throughout 
the world were made dramatically aware of the need to develop 
simulators capable of creating, in the gravitational field of the earth, 
the conditions and problems that astronauts would encounter in leaving 
that field, living from hours to weeks in a very confined area imder 
conditions of zero gravity and finally re-entering the once familiar 
environment The resulting simulators represent engineering feats that in 
some respects rival the space ships themselves. Yet no one questions 
that these costly technological marvel were indispensable: first, in 
assisting scientists to anticipate and work out alternative solutions to 
problems in a deliberate, step-by-step fasMon, free of any pressures that 



r!j!^w2f%i?*'*^' Educttlonal D^yelopmcat, Unltcnlty of niinoU College of 




Medicioet Chloifo, HL, USA. 
* Fonnerhr. Center for VAtu 



18 




ERIC 



time and danger ordinarily impose; and second, in providing space-fiig|it 
crews with ample opportunity to practise these solutions in a risk*free 
environment, so that all could be assured of appropriate responses in 
the potentially dangerous situation of the real universe. 

But space simulators for training astronauts are merely the most 
recent and techmcally the most sophisticated of a growing number of 
devices that have long enjoyed a deservedly favoured status as instruc- 
tional adjuncts in the education of such diverse personnel as athletes, 
business executives, lawyers, and many other groups. However, despite 
widespread fomiliarity with the principles that underlie these intriguing 
approaches, and almost universal acknowledgement of their demon* 
strated effectiveness, Emulation technology has only recenfly been 
introduced in medical education. Perhaps such limited expldtation to 
date is in part owing to the vision of astronomical costs conjured up by 
the word ^^dmulator^. However, recent developments suggest that^ 
within reasonable financial limits, this exciting tedmology can be made 
available to medical educators to enhance the efficiency and effectiveness 
of learning and the validity and reliability of assessment 



Reduced to its essence, simulation consists merely in placing an 
individual in a setting that imitates some aspect of reality, and designing 
a problem around that setting which requires that learner's active 
participation in initiating and carrying through a sequence of inquiries, 
decisions, and actions. The total situation must be arranged so that 
each of the learner's activities triggers appropriate feedback that he can 
utilize in subsequent decisions about pending action — decisions that 
may in turn modify the problem. Certainly one of the most important 
characteristics of simulation is that a problem can be initiated in 
standard form for everyone but evolve in quite different ways depending 
on the imique configuration of interventions that each respondent makes. 

In continuing experimental work (Fig. 1) this essence has been 
captured in a variety of sunulation modalities whose use is economically 
and technologically feasible in the development and assessment of 
critical components of physician competence. 



The various types of sunulation outlined below are based on a 
recognition that clinical and laboratory data come to the physician 



THE ESSENCE OF SnCULATION 



TYPES OF SIMULATION IN CURRENT USE 



Id 
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FIQ. 1. GROUPS EXPERIMENTING WITH SIMULATION 



The foHowing groups are currently engaged in experimental woric 


with one or more types of simulations for Instruction or evaluation. 


University of Illinois 


University of Southern California 


College of Medicine, 


School of Medicine, 


unicago, iii., usa 


LOS Angeles, uaiir., usa 


University of Miami 


University of Iowa 


School of Medicine, 


School of Medicine, 


Miami, Fla., USA 


Iowa City, Iowa, USA 


Michigan State University 


McMaster University 


School of Human Medicine, 


Faculty.of Medicine, 


East Lansing, Mich., USA 


Hamilton, Ontario, Canada 


The Royal College of Family . 


The Australian Academy of 


Physicians of Canada, 


Genera] Practice, 


Toronto, Ontario, Canada 


Melbourne, Australia 


The American Board of 




Orthopaedic Surgery. 




Chicago, IIL, USA 




At WHO headquarters, Geneva, Switzerland, a demonstration 


laboratory of instructional aids Including simulations has been estab- 


lished under the direction of Dr. M. A. C. Dowling and certain lnte^ 


country studies initiated on the relative efficacy of some of the simulation 


models described in this paper 





through various sensory modalities (eg., visual, auditory, tactile) and 
that highly complex cognitive and psychomotor skills are involved in 
collecting and processing such data. These important facts are com- 
monly ignored in much of the teaching and most of the testing 
characteristic of medical education, where all too frequently almost 
exclusive reliance is placed on verbal descriptions of highly complex 
and varied phenomena and processes. 

Reproductions of visual and auditory stimuli 

The utility of photographic reproductions of X-rays, lesions, gross 
and microscopic specimens, tracings of electrocardiograms, or electro- 
encephalograms and the like — all presented in a form that imitates life 
as closely as possible — has long been recognized. More recently 
electronic techniques have been developed for the playback of high*- 
fidelity recordings of heart, breath, and abdominal sounds using 



individual stethophones in a manner that simulates what the physician 
hears during h^ examination of a patient Similarly, high-quality 
cin6-films or videotapes depicting patient interviews and exammations 
arc generally recognized as invaluable for the presentation of data 
consisting of a combination of sound, colour, and movement 

For purposes of instruction, presentation of data in these forms 
makes it possible to control the difSculty of the material and to enable 
a student to focus on the essential elements, freed from the multiplicity 
of distractions that real life intrudes. For purposes of assessment data 
presented in this manner can be employed in standardized test exercises 
that require the student to demonstrate th. t he has observed the salient 
findings and can describe them accurately, interpret their significance, 
and anticipate other findings to which they may be related While such 
methods are being increasingly used in the instruction of the medical 
student, their utility in the evaluation of the students competence is 
only now beginning to be recognized and is, as yet, inadequately 
exploited. 

Three-dimensional models 

More recently, prototypes of several three-dhnensional simulators 
(e.g., breast, pelvis, prostate, eye, ear, larynx, infusion arm, intuba^on, 
spinal tap, etc.) have become available.^ Varied pathology can be 
inserted in these devices to assist the student in learning and to 
require him to demonstrate that he has mastered the skill of using 
sophisticated instruments to elicit the findings and to interpret thent 

For example, the Iowa Eye Model consists of a hollow styrofoam 
head with realistic, transparent cornea and pupils. The interior of the 
head is fitted with slots into which ordinary 35-mm slides of the fundus 
are inserted in front of a silvered surface designed to reflect the light 
from an ophthalmoscope back through the 35-mm transparency in a 
manner that enables the physician to visualize the retina, much as he 
would in a real patient Since standard 3S-mm slides are used in a 
manner that makes them readily interchangeable, it is obvious that the 
findings in this model are capable of almost infinite variation* This 
important feature means that for purposes of instruction the findings 
can be carefully graduated in difSculty, while for purposes of assess- 
ment they can be selected to require the student or physician to 
demonstrate that he can make interpretations ranging from the gross 
distinctions expected of the be^nner to the most subde expected of the 
specialist 



2 Peiita» F. A Kofman, S. (1973) /. med. Educ., 4t, 442. 



SI i 
• % 



Automated robots 

As an indication of the level of realism that can even now be 
attained in three-dimensional models, SIM I (developed at the University 
of Southern California for use by anaesthetists) and the new Cardiac 
Simulator (developed at the University of Miami) may be instructiv©. 
SIM F is a computer-managed, extraordinarily life-like robot that can 
be programmed to present combinations of findings capable of modifica- 
tion in an almost infinite number of ways, to simulate a variety of 
problems, and to respond appropriately to different interventions of the 
anaesthetist SIM I may even "die'' if the student manages it 
inappropriately or responds inadequately to changes in its condition. 
The Cardiac Simulator,' although not computer managed, is equipped 
with a large number of programmes presenting varied auscultatory 
findings coordinated with appropriate visual pulses built into a life-like 
torso. 

While these robots are at present available only in prototype form, 
they suggest some of the possibilities for using sophisticated simulation 
technology in the training and testing of medical students and 
residents. 



Paper-and-pencil simulations 

At the other extreme of the cost continuum are the written 
simulations of clinical problems,' widely known as branched problems 
in patient management (PMP), designed to imitate the decision-making 
process involved in the **work-up" and management of a patient In 
Ihese problems a clinical situation is introduced with a brief statement 
of a complaint such as a physician might elicit from the patient himself, 
from a friend, from a relative who . accompanied hhn, or from a 
colleague who referred him. It may consist of a few written lines or a 
brief length of a film in which the patient indicates his reason for 
"seeing the doctor**. 

The statement of the problem is followed by a list of interventions 
or of general strate^es; from these an initial approach may be selected, 
e.g., further history, physical examination, laboratory evaluation, or 
immediate therapy. This decision is reconled by erasing the opaque 
overlay or developing the latent image on a specially treated answer- 
sheet to reveal the requested information or further instructions. 



> AbrthMMon, S, et al. (1969) /. mtd. Educ,, U, 515. 

* Oordon, M. (1971) Compuitrtx/td aitomated matmemdn, a ttlf-tgstint method (ucptiblished. 
sraUable tnm the Univenity oC FlorliU School of Medichie» liltmL FU.» USA). 

* UeOvkt, C. & Solomon, ed. (1971) CltiUca simuMons:iilgci€d problems In mtflmr 
numagementt New York, Appktoo-Centnry-Crof ti. 
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In each new section possible interventions that may yield farther 
itiformation about the patient or modify the patient's condition are 
listed. The respondent selects as many or as few as seem appropriate 
in light of the specific circimistances obtainhig at that stage of the 
proUem. As each decision is recorded the results of that iniervention 
are revealed in realistic verbal or visual form. As the problem unfolds, 
the student must make further choices based on the specific responses 
of the paient evoked by his own eaiHet decisions* The complications 
that the must manage wQl depend, as they do in the consulting room or 
clinic, on the unique combination of specific procedures that ^ere 
selected at earlier stages* \' . 

This type of paper-and-pencil simulation has been found suitable for 
both diagnostic and management problems ranging from emergency, to 
long-term chronic disease conditions that may involve psyclibsociii ^^ 
well as organic factors* Recent technical developments make it possible 
for the local faculty to devise such simulations and to reproduce smalU 
quantities of them in latent image by means available in any secretarial 
oflSce and at a cost comparable with other ines^nsive duplicatii^ 
procedures*^ . , % 

Computer-mded simulations , . 

A rationale analogous to that used in the construction of written 
simulations has been employed in the development of a variety of types 
of computer*assisted Emulations of the clinical encounter* Altbou^ the 
specific details vary, in essence all are designed to imitate seme* aspect 
of the decision-making process in the investigation and management 
of patients* 

In some such simulations the student makes selections from a list' 6f 
options presented by the computer; in others he must formulate his 
own inquiries and interventions with neither the assistance nor the 
constraints of a list For example, in the CASE format, developed at 
the University of Illinois,^ the computer takes the role of a patient 'and 
the session is introduced by a computer statement describing the settiUg 
(physician's consulting room, hospital, emergency room, etc.) and the 
patient's age, sex, and genersd appearance. The computer may go on to 
state, as a patient wouM, the reason why the patient has come to see ibe 
physician* Or alternatively the session miglit simply begin as follows: 

Student: What brings you to see me? 

^ llcOuin* C. et al. (1974) Written timuimiom: their eonstrudUm emd muesli (ttopabUihcd, 
available from Center for Educational Developmeot, University of Illinois Colleie of Medicine, 
Chlcaio. St. USA). 

> Harioe, W. ct al. (1971) med. Eiuc^ U, 443. 
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Computer: Well, I am just tired all the time* I don't ever feel rested 
and all my energy is gone* 

Student: How long has this been going on? 

. Computer: About six months ago I started feeling tired all the time. 
I finally felt I had to do something about it 

Student: What did you do? 

In this fashion, the student may question the computer-patient about 
the history and presenting complaints and the computer responds as 
the patient would At any point in this interaction a physical examina* 
tion or laboratory evaluation may be initiated In response to each 
specific inquiry tiie computer provides a technical description of the 
physical findings or the laboratory report requested. At ti^ condu^on 
of this data collection tiie student records both his diagnosis and his 
plan of management At this point the computer assumes the role of 
a oMisultant imd immediately responds witii a display comparing the 
data actually gatiiered witii the optimal data base and tiie proposed 
diagnosis and management plan witii the consultant's conclusions and 
recommendations. 

In an alternative type of computer programme for simulating the 
patient-physician encounter, deveibpcd at the National Institutes of 
Healti),^ the student n ust also manage simulated time and cost in a 
realistic manner. Here, the task is to arrive at the correct diagnosis 
witii a minimum appropriate expenditure of time and money. At any 
point in the session the diagnosis may be entered; the computer will 
tiien respond with an evaluation of its accuracy and a summary of 
the time and money cost incurred compared with the cost of the 
optinium approach* 

Irrespective of their specific focus and methodology, all computer- 
assisted simulations ars designed to prcvide the student with 
instantaneous feedback on the results of his decisions and an 
immediately retrievable record of his entire decision-making sequence 
for objective analysis and evaluation. 

(.ive simutattons 

Realistic clinical problems can also be posed through live simulations 
in wluch an actor, housewife, anotiier student, or ahnost anyone can be 
^programmed'' to simulate a patient in an interview settmg. In 
principle, live simulations differ from written and computer problems in 
only two respects: first, the student/physician must necessarily generate 



t Frkdauui, ILA. (1973) 7. rwd. E4uc., 41, 92. 
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his own iaqoirics and interventions; and second, he must formulate 
them in a manner that is meaningful to the '"patient* \ 

In a emulated diagnostic interview,^ for example, the student may be 
directed to interview the programmed patient and elicit data from him; 
in a management or tiierapeutic interview it may be tiie student's task 
to persuade Uie nmulated patient to accept a recommended plan of 
management In either case, the emulated patient is programmed to 
display the reluctance, anxiety, and need for understanding and 
emotional support that such patients dis{tey in real life* And it is the 
student's task to communicate clearly and convindngly widi the patient 
in order to gain his fidl cooperation. 

Skills in data collection, long-term care, or crisis management have 
been explored by means of a variety of such, techniques. In a 
telephone shnnlation, for exan^le, tiie student is seated hi a setting 
similar to a consulting room when the telephone rings and a frantic 
voice cm the other end of the Ime reports some emergency— -an accident, 
a child suspected of ingesting poison, a suicide threat, or the like* The 
physician is required to respond as he would to a reid-life emergency 
and tile adequacy of his response can be reliable analysed, eiUier for 
purposes of certification or for further learning. 

live simuUtioQS have also been used to teach and to assess various 
fViiif in patient examination. Of spedal note in this regard is the 
experience of Dr Howard Barrows and his associates in programming 
simulated patients to display the physical findings of various neurolopcal 
disorders.' 

Similarly, the physician's interpersonal skilL and attitudes can be 
enhanced as well as assayed by carefiilly designed simulated inter^dews 
and conferences in which an individual — again, not necessarily a 
professional actor — is programmed to take the part of a patient, a 
colleague, or otiier member of tiie healtii team.* Such intendews can 
be developed to focus primarily on the student's sensitivity and skiO in 
handling various types of patients; on important referral, consultation, 
and conference skills, including those required in the delicate situations 
in ^ch colleagues differ sharply about tiie management of a particular 
patient; or on skill in communicating with other members of tiie health 
team (e.g., instructing a nurse, requesting assistance from dietitian or 
social worker, or making a presentation in a staff conference). Such 
simulations are useful in as^ting the student to develop, and his 
mentors to evaluate, the range of communication and interpersonal 
skills essential to effective functioning as a physician. ' 
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Future dtrecHons 

ht short, reproducttons of clinical and laboratory data, threc-dtmeii- 
sioMl models, automated robots, writtea simulations, computer 
simulations, and live simulations can all be used to present clinical 
problems in a realistic fonn that («) enables the student to learn, and 
requires him to demonstrate that he has learned, how to respond as 
he would in the consulting-room or clinic setting; (b) permits him to 
receive lif e*Uke feedback from the simulator, and (c) foices him to cope 
with the resulU of a sequence <^ inquiries and interventiotts, much 
as he will have to do in his actual practice. 

Though simulation technology is still relativeiy new to medicine these 
methods arc already being employed in teaching and testing medical 
students, residents, and clinicians. It may in the near future be posuUe 
to combine various simulation modalities in a manner that further 
enhances then* utiliQr. For example, a computer simulation could be 
merged with a live simulation so that a constantly updated computerized 
daU bank is made available to supi*y infoi-iation about the pathdogy 
(eg,, lAysical and laboratory findings) that alive simulator is incapable 
of providing Such a simtdation could be extended to cover the full time 
period beginning with the presentation of the patient and concluding 
only after appropriate treatment has resdved the problem and the 
evident changes in the "patientV* condition have met suiuble criteria 
for discharge. In such a problem changes over time in the patient's 
condition induced by, and resulting from, intervention by the lAysidan 
would be reflected not only in modified findings supjA'ed by the 
computerized data, bank but also in changes in the spec^ roles and 
reactions of the simulated patient and of the f^ysician* Tlius, a complete 
clinical encounter could be simulated in a condensed time period But 
merging live and computer simulations is only one of the many possible 
ways of combining all types of simulation modalities to increase their 
versatility and effectiveness in medical settings. 



THE USB OF SIMULATIONS 

In instruethn 

As indicated above, the various simuhition methods now available 
in medicine can be employed as instructional resources not only in the 
development of diagnostic and proUem^ving skills, but also in the 
development of psychomotor and interpmonal skills. Qearly, these 
simulations can provide more valid educational experience than man/ 
erf our conventional modes of instruction. However, this increased 
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validity depends upon the fidelity of the simulation, whether it is 
presented in a live, (XHnputer, paper*and-pencil, or dthea: fcmnat While 
fidelity is a relevant consideration in all Emulation methods, the 
problem is particularly acute in the development of three-dimensional 
models, in which it has been especially ditBcidt to simulate tissue 
characteristics accurately. With the advent of improved materials and 
better production techniques it has now become possible to develop 
sufficiendy accurate represencations of human anatomy for teacUng 
some of the skills of physical diagnosis. Analogous advances in 
electronics have led to the development of more compact electronic 
Emulation devices (such as the Humetric Sound Device) that 
electrraically synthesize heart soimds and other phenomena in the body. 
These, too, have proved of great value in assisting medical students to 
develop sldll in interpreting certain types of physical findings. 

Such innovations afford more efficient means of (a) focusing upon a 
specific leammg objective; (b) rendering the experience reproducible; 
and (c) allowing a standardized approach for all students. In the use 
of simulation devices it is important to keep clearly in mind the nature 
of the desired behavioural outcomes, smce the ultimate goal is the 
subsequent independent performance of the skill in a non-amulated 
setting. The dega^e to which a learned sldll is transferable from a 
simulated setting to a live patient situation depends not only on the 
fidelity of the device itself but also on its incorporation into a systematic 
total educational approach. Mere exposure to a device affords the 
student a chance to experiment with or interact with the device only 
at a superficial leveL On the other hand, the development of assodated 
programmed instructional guides that direct the student— device inter* 
action magnify the relevant aspects of the experience. Thus, statements 
of the goals and objectives of the experience with the device, 
performance skill expectations, methodolo^cal uistructions, and 
evaluation items expedite the acquisition of the sldll, direct the 
interaction with the device, and guarantee a profitable experience. 



In assessment 

The results of current research sugg^t that simulation techniques of 
the types described here are especially useful in evaluating (a) skill in 
gathering and interpreting clinical and laboratory data; (b) judgement 
m patient mana^ment; and (c) the professional skills and attitudes 
required for effective interaction with patients and colleagues (Table 1). 

Interpretative skills. Qearly, greater reliability can be placed on any 
judgement about a student's interpretative skills if he asked to interpret 
data presented in the form in which he usually encounters them rather 



TABLE 1. SIMULATION AS AN EVALUATION TECHNIQUE 



Compftenct t6»t«d Typ« of timutatlon Illustrative technique 



Interpretative skills Simulated clinical Photographic reproductions 
and lat)cratory Sound simulators 
data Cin6-films and videotape 

ThreeKlimensiona! models 
Automated rotx)t — SIM I 



Problem-solving Simulated problems Written simulations (PMP) 
skills and clinical in patient Computer-assisted simulations 

judgement management (CASE) 

Automated robot simulations 
(SIM I) 

Live simulations (telephone, sur- 
gery, etc.) 

Interpersonal skills Simulated interviews Programmed patients 
and attitudes and conferences Programmed colleagues 

Programmed teams 



than verbal descriptions of such data. For this reason, the use of 
photographic reproductions, sound simulators, motion pictures and 
videotape, three-dimensional models, and more sophisticated robots are 
of particular use in assessing this important aspect of clinical 
competence. 

Problem-solving skills and clinical judgement Patients ordinarily 
present with a chief complamt, which the physician must explore and 
manage. Thus there is a certain "face validity'* to any technique that 
confronts the student with a problem in this form, provides him with an 
opportunity for considerable latitude in choosing his own approach, 
and enables him to leceive prompt feedback on the results of any 
inquiry or intervention he makes. Qearly it will be more useful in 
assessing the student's clinical judgment than the more traditional 
exercises that ask him to list the signs and symptoms of various 
diseases, and the most probable diagnosis from a carefully predigested 
summary of the salient data. 

Professional skills and attitudes. Too often in the past, medical 
educators have either failed to evaluate psychomotor and interpersonal 
skills or have attempted to make inferences about these essential 
components of competence from the student's verbal description of his 
interaction with a patient or from the written record of a patient 
investigation. Clearly, both sources of data are inadequate for a valid 
assessment of skills and attitudes. These can be judged only by observing 
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the student with a patient However, since the student may behave 
differently with different patients and since patients must not be put at 
risk from incompetent interventions, observation of the student with 
simulators, both mechanical and live, would seem to be the most 
promising alternative. 



CRTIERIA FOR USING SIMULATION 

/n instruction 

Certain criteria for the use of simulation devices have been implied 
in the preceding discussion; it would be possible to identify others that 
are especially pertinent to the instructional uses of simulators. To 
catalogue them, however, would be umewarding since many relate 
merely to common sense and good judgement First among these is the 
requirement that the development and use of a simulator parallel 
curricular needs. This assurance, together with the advice of members 
of the teaching faculty, guarantees that experience with the device 
will be a supportive extension of the stated curriculum, as well as 
one that gathers support for further application of such aids to the 
teaching programme by tho faculty members themselves. 

No matter how clearly the experience coincides with stated curricular 
needs, or how apparenfly acceptable it may be to teaching faculty and 
students, other sources of difficulty may jeopardize the validity of an 
experience with simulators. Such problems may arise from the physical 
nature of the devices themselves. In the area of anatoniical replications, 
commercial development hi the past has poorly satisfied the need for 
instructional accuracy in terms of texture, size, apparent "tissue'' 
density, or mobility. The resulting lack of structural reality sets an 
mcomplete stage for the mteraction of the student with the device. 

In addition, careful attention must be given to the level of education 
in question, the placement of skill leammg in the educational con- 
tinuum, and the knowledge that must be attained before experience with 
a specific simulation device is of maximum benefit Obviously, the 
sunple skill of perf ormmg a pelvic examination can be acquired without 
understanding the normal or abnormal pathology and physiology of the 
female reproductive system. Appreciating the importance and value of 
either the simulated or real experience demands a knowledge of basic 
sciences, pathology, and some fundamentals of obstetrics and 
gynaecology. Thus, it is of greater value to expose a student to a 
simulated learning device at an appropriately defined time in his 
traimng. 
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In assessment 

In selecting simulation techniques for purposes of evaluating 
individual competence and/or curricular effectiveness, the same con- 
siderations apply as in the choice of any other testing and evaluation 
techhique. As with the instructional uses of simulation, it is first 
necessary to decide exactly what aspect of competence is to be 
measured at each level of education and experience. Once this decision 
has been made, then the most appropriate technique is that which 
requires the student to perform a task resembling as closely as possible 
the real-life performance for which he is being examined (test validity). 
Secondly, it is necessary to deagn the task in such a manner that 
different observers who are evaluating the student will agree about the 
character of his performance in the test situation (objectivity or inter- 
rater reliability). Third, it is essential to create a sufficient number and 
variety of tasks to ensure that one can safely generalize from the 
student's performance on the test exercises to his overall level of 
competence (sampling reliability). Finally, both economic feasibility 
and logistic considerations must be taken into account in selecting 
any assessment technique. These four criteria should be borne in mind 
in reviewing the advantages and limitations of the simulation tech- 
niques elaborated upon below. 

ADVANTAGES AND UMrTATIONS OF SIMULATION 

Simulation imitates, it does not duplicate, life; this is both its greatest 
advantage and, in the view of some, its greatest limitation. Clearly 
there are some aspects of reality that cannot at the present time be 
economically simulated, if they can be simulated at all. Thus, it is 
necessary to be very cautious about predicting on the basis of responses 
to a simulated situation how an individual will behave in reality; at best 
it is possible to predict how well that individual is capable of behaving. 
(The same caution must, of course, be observed in predicting how an 
individual will behave in a new life situation on the basis of how he 
behaved in a prior, necessarily different, life situation). 

Furthei, it is important to recognize that simulation is not an 
appropriate method for teaching or for measuring all aspects of 
performance. For example, factual information is more economically 
conveyed and more directly measured by conventional techniques of 
instruction and testing. On the other hand, appropriate professional 
habits are most firmly entrenched by repeated reinforcement over a long 
time-span in diverse settings; they are most reliable assayed by careful 
and repeated observation under similar conditions. Between these two 
extremes simulation provides the following important advantages. 
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Perceived relevance. As compared with more conventional teaching 
and testing techniques, simulation offers the possibility of designing 
instructional and assessment exercises so that th^ correspond more 
dosely to the situations a physician actually faces. Such materials 
appear to the student as far more relevant than typical readings, 
lectures, multiple-choice tests, or traditional oral quizzes. Such perceived 
relevance is at the very least of psychological and motivational benefit 

Predetermination and preselection of the task. It is important to note 
that this relevance can be achieved without dq)endence on the flow of 
patients available at the particular place and the specific moment in time 
that instruction is to take place or assessment is to be made. Thus 
simulation makes it possible to predetermine precisely the exact task 
that students are to learn or to demonstrate that they can perform. 
Further, by suppressing the undesired distractions always present in 
reality, it is possible in a simulation to focus on the elements of primary 
concern and to eliminate inelevant, confusing ' complexities that 
complicate learning and contaminate assessment 

Hie fact that simulated tasks can be preselected also means that they 
can be carefully graduated in difficulty as a student or resident 
progresses through training — a factor of prime significance in 
increasing the efficiency and effectiveness of both instruction and 
evaluation Hnally, by developing standard parallel sunulations it is 
possible to confront a student again and again with challenging varia-- 
tions of what is essentially the same task until he has mastered it and 
can demonstrate that mastery to any examining body. 

Standardization of the task. Just as a single student can be repeatedly 
confronted with the same task, simulation enables an CAamining bod/ 
to standardize the task for all examinees, and to do so without 
subjecting one or a few patients to repeated harassment by a large 
number of students. In short, all examinees can be given exacdy the 
same problem to cope v,1th and this can be accomplished without risk 
to any patient 

Improved sampling of performance. By standardizing the tasks and 
focusing on the most significant aspects in each it is possible in a given 
time period to expose the student to, and to sample his performance on, 
a much broader and more representative group of pioblems. This, of 
course, requires careful selection and definition of the tasks, but that is 
precisely the luxury sunulation affords that reality can rarely provide in 
a reasonable time frame. 

Improved rating of performance. When the exact tasks that are to 
compose an instructional unit or an evaluation exercise are precisely 
defined and preselected it is possible to record the student's 
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performance in detail for subsequent feedback in a manner that 
maximizes his learning. Further, for piuposes of evaluation it is possible 
to develop specific criteria for jud^g performance, and to train 
examiners in applying these criteria equitably and consistently. Thus the 
degree of inter-rater reliability that can be achieved m scoring examinee 
performance with simulated problems and simulated interviews is far 
superior to that achievable with real patients. 

Increased responsibility and realistic feedback in a practical time 
frame. One of the most important advantages of sunulation over reality 
consists in the fact that all students and residents can be allowed full 
responsibility for the work-up and management uf the simulated patient 
without any risk whatsoever to actual patients. Even if the student's 
interventions with the simulated patient are grossly inappropriate he 
can be allowed to continue without supervision, to receive realistic 
feedback, and to learn the consequences of his action from first-hand 
experience, without harm to anyone. The simulated patient can develop 
complications induced by faulty management, get very much worse, die, 
commit suicide, and yet be repeatedly revived, tirelessly to serve in the 
edification and examination of the same or other students. 

Compression of real time. Furthermore, it is often essential to provide 
a student experience with, and to test his mastery in the handling of, 
chronic disease without his spending months or years observing the 
gradual deterioration of a real patient. In carefully developed simulations 
a lifetime of chronic disease can be compressed into u half*hour 
problem, at each stage of which the student can be provided with 
feedback about his interventions in a form that is more instructive 
than life itself usually yields. 

Increased learning. The nature of the feedback provided in simula- 
tions as compared with that available in either conventional educational 
settings or in reality should be stressed. While the former rarely provides 
any genuinely relevant feedback, the latter often furnishes it only 
indirectly, equivocally, and after a long delay. In contrast, the prompt, 
specific, and unambigious feedback characteristic of well-^signed 
simulations makes them a powerful tool for the enhancement of 
leammg. 



RELATIONSHIP OF SIMULATION 
TO OTHER INSTRUCTIONAL AND EVALUATION STRATEGIES 

Enumeration of the foregoing advantages should not be construed as 
a recommendation that conventional, instructional and evaluation 
stratepes be entirely replaced by simulation techniques; nor should it 
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be interpreted as suggesting that simulation is a pale, but adequate, 
substitute for the real thing when the latter is unavailable. Rather, 
simulations should be viewed as important adjuncts to other educa- 
tional methods. Their power lies in the fact that they select out 
manageable segments of the total physician/patient relationship for 
study. However, tlus very fragmentation, in turn, requires that provision 
be made for the student to experience through conventional patient 
contact the total interaction at appropriate points in his professional 
development 



CONCLUSION 

In summary, the evidence from studies of the use of Emulation at 
several levels of medical education and with a variety of types of 
groups suggests that when these techniques are properly exploited, 
simulation technology gives considerable promise of increasing the 
effectiveness of instruction and the validity and reliability of assessments 
of more relevant and complex objectives of medical education. Simula* 
tion appears to offer special advantages as an effective technique not 
only in teaching and assaying specific, circumscribed skills but also in 
developing and evaluating that most significant objective of all — 
clinical judgement 
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EXAMINATIONS AND DECISION-MAKING 
James A. Bobula^ 



One of the most challenging tasks facing medical educators is that of 
developing exammations that measure accurately and assess appropri- 
ately the adequacy of student knowledge, skills, and professional 
attitudes. This evaluative task is of such importance because it is chiefly 
through examination data that teachers monitor student progress 
through medical school, identify those who are to qualify as physicians, 
and thus ultunately control both tlie quantity and quality of the 
physicians available to society. This paper de^ m an introductory, 
non-iechnical fashion with the systematic planning, scoring, and 
reporting of examinations to serve those critically important functions. 

No single examination can provide adequate data for all the decisions 
a faculty must make about medical students. Each examination, how- 
ever, is most likely to sei-ve a particular evaluative need if it is designed 
from an explicit statement of the decision a faculty must make and is 
developed to provide the specific data reqmred for that decision. For 
example: 

A Proficiency Examination will be administered to incoming first-year students 
to determine whether any student has sufficient mastery of: (a) one or more 
disciplines within the basic science curriculum to warrant either placement into 
advanced courses or exemption from regular courses m such disciplines; or 
(b) the basic medical sciences as a whole to warrant pla^^ement directly into a 
clinical curriculum. 

Given such a statement of purpose, the particular examination 
characteristics that are consistent with the decision to be made must be 
specified and the exammation planned on the basis of those specifica- 
tions. The example summarized in Table 1 is further developed below. 
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TABLE 1. EXAMINATION PLAN 



Examination characttrlttict 



Proflcitnq examination examptt 



1. Scope of material covered 



The content will systematically represent the 
entire basic science curriculum, according to 
a blueprint developed by a special faculty 
committee composed of one representative 
from each basic science discipline. 



2. Format Multlple-cholce questions with only one 

correct answer will be employed exclusively 
In the examination. 

3. Nature of data A student will receive a score Indicating the 
(I.e., type of scores) percentage of his answers that were correct 

in each discipline and In the examination as 
a whole. 



A minimum passing percentage score will be 
established by the faculty committee prior to 
the examinatlort for each discipline and for the 
total examination. A student will be considered 
at least minimally competent In a particular 
discipline or In the basic sciences as a whole 
if he obtains a percentage score equal to or 
greater than its corresponding minimum 
passing percentage score. 

5. Critical review The faculty's Appraisal Committee will review 

the examination blueprint prior to administra* 
tlon of the examination to approve Its 
specifications. After the examination Is ad* 
ministered, the Committee will assess the 
reliability (Internal consistency) of the exami- 
nation and will perform an Item analysis to 
determine whether any of the multipie*cholce 
questions were ambiguous to the students and 
must be deleted from scoring. 



4. Basis for decision 
(I.e., standard of 
adequate performance) 



6. Reports A report of each student's percentage scores 

and the minimum passing percentage score for 
each discipline and for the total examination 
will be made available to the student himself, 
to his faculty adviser, and to the faculty com- 
mlttee responsible for deciding on advance 
placements. 
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THB DEVELOPMENT PROCESS 



Step 1: state the ptirpose 

A statement of purpose should indicate the reason for whirJi an 
examination is being constructed and administered, specifying (a) the 
group of students to be examined, (b) the are^s on which they are to be 
examined, and (c) the decision that the staff must take on the basis of 
the examination results. 

Specification of the group of students to be examined enables 
definition in broad terms of the skills required (e.g., students at the end 
of the first year and of the fourth year mig}it both be expected to know 
something about cardiac arrhythmias, but at quite different levels of 
competence). Specification of ^e content about which the students are 
to be examined tells ^*hat particular areas are to be dealt with in this 
e:^amination. Specification of the decision to be made about the 
students indicates how the examination data are going to be used in 
evaluating student performance, i.e., to monitor a student's progress 
(diagnostic examinations) or to judge competence at some significant 
transitional point (certifying examinations). 

Diagnostic examinations assess mastery of a specific unit of instruc- 
tion so that a decision may be made whether or not further learning 
experiences are necessary. Hieir purpose is to advise students in 
planning subsequent learning activities. Diagnostic examinations include 
pre-tests to determine what a student knows about certain topics before 
they are formally treated in the curriculum, mid-unit tests to assess 
progress during instruction, and post-tests to determine whether the 
expected levels of proficiency have been achieved by the end of an 
instructional unit 

Certifying examinations, on the other hand, assess mastery of an 
entire area of study so that a decision may be made on promotion, 
graduation, or licensure. Certifying procedures include placement 
exaninations» which permit a student to bypass formal instruction in 
areas that he has already mastered; comprehensive examinations, which 
document a student's achievement of professional competence at the 
end of a formal course of study; and licensing examinations, which 
testify to the ri^t of professional practice. 

Step 2: determine what is to be examined 

In order to achieve a specified purpose, it is necessary to develop a 
(ktailed blueprint outlining both the content of the examination and the 
performance processes the students are expected to demonstrate with 
respect to that content "Contenf * refers to the body of knowledge, set 
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of skills, or focus of attitudes on which the students are to be examined. 
"Performance processes" refer to what the students are expected to be 
able to do with the content being examined In the area of cognitive 
performance processes, for example, students might be expected: (1) to 
simply recall information ("What is X?"); (2) to interpret data using 
this information ("Knowing what X is, what do these data mean?"); or 
(3) to solve a problem by fitting this information together with other 
information in reaching a conclusion about a complex phenomenon 
(''Knowing the nature of X and its relationships to A, B, and C, what is 
the nature of Z?"). 

An examination blueprint is best presented as a table of specifications 
that lists relevant content areas on one axis, relevant performance 
processes on the other, and the proportion of the examination to be 
devoted to each content/process combination within the cells (Table 2). 
The proportions generally reflect the relative importance of these 
elements: more important areas are given greater weight in the examina- 
tion; less important areas receive a smaller portion or are eliminated 
entirely. 

TABLE 2, EXAMINATION BLUEPRINT : CCGNriVE DOMAIN 

Performanco processes 



Content areas 

Area^i 
Area 2 
Area 3 
Area 4 
Area 5 



Vo of 
examination 

20 
35 
10 
20 
15 



Recall 
30 

12 
5 
3 
8 
2 



InterpretaUon 
40 

8 
10 

4 

6 
12 



Problem'Sotving 
30 

0 
20 
3 
6 
1 



The development of a blueprint and preparation of test materials 
according to that design increase the probability that examination's 
content will be consistent with its purpose and that the performance 
data will provide a sound base on which to make the decisions for 
which the examination is intended. 

Step 3: determine the format 

"Format" refers to the types of examination materials and methods 
selected to measure the specified contents and processes. Formats are 
broadly categorized as "free-response** or "fixed-response** types. Free- 
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response methods, which allow a student to respond to a question or a 
situation with his own words or actions, include oral examinations, 
extended-response written questions (essays), and short-response written 
questions (sunple direct questions, sentence completions). The 
unstructured description of student behaviour by faculty observers is 
also a free-response type, although in this case the response is made 
by the observer and not by the student himself. Fixed-response 
materials, which require adherence to a predetermined form, include 
true/false, multiple-choice, and matching questions, patient management 
problems, and checklists or rating scales completed by examiners 
observing student behaviour in cither an oral exainiuuiion or a natural 
performance setting. 

Free-response examinations have the advantage of requiring a 
student to respond independently, drawing solely on his own resources. 
They have the disadvantage of being virtually impossible to correct and 
score mechanically and are often subject to such extraneous influences 
as an examinee*s general verbal fluency, an exammer*s lack o£ clear 
criteria for essential elements of a correct response, or differences in 
judgements made by various examiners. Fixed-response examinations, 
on the other hand, can be scored mechanically or by any examiner with 
a listing of the correct responses. Their major disadvantages are that 
students might respond correctly by guessing or might be cued to correct 
responses by the options offered Since skilful development of fixed- 
response formats allows testing of many complex as well as simple 
objectives of medical education, the additional advantage of objectivity 
and ease of scoring recommend them highly for most examination 
purposes. 

Selection of a particular format should be determined primarily by 
the degree to which it requires a student to perform the desired process 
in the specified content area. For example, when the purpose of an 
examination is to Sisstss the student's competence in obtaining a 
patient's medical history it is more appropriate to have a trained 
examiner with a fixed-response checklist observe him taking a history 
than to ask him, "What questions should be used routinely in obtaining 
a patient's medical history?" 

Finally, an examination need not be limited to a single format. The 
blueprint can prescribe the use of different formats within an examina- 
tion wherever they are most appropriate. 

Step 4: develop a scoring system 

The purpose of a scoring system is to summarize performance in 
some symbolic form that can serve as a basis for rational decision about 
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the adequacy of that perfonnaoce. Selection of the specific method 
ihould be^ with a dear statement of exactly ^^t the score is intended 
to mean. Such a statement might read: student's score in each 
disdidine included in the Profidency Examination will indicate the 
percentage of the total content of that disdpline that he has mastered'' 

In ord;^ for a score to have tins meaning, the examination blueprint 
must ensure that the total content of each discipline is systematically 
represented Unless it is the score cannot indicate a mastery of the totid 
discipline but only the components that were included 

Assuming that the blueprint supports the scoring purpose, it will then 
be necessary to assign some numerical value to each response a student 
makes. Since the form of the response depends upon examination 
f ormat, assignment of values must be consistent with that format If, for 
example, multiple-choice items having a sin^ correct response are 
employedt then a correct response might be given a value of 1 and 
any other a value of 0, Multiple-choice items having several coaect 
options would require a different procedure. For example* with three 
correct responses each might be assigned a value of 1 (so that a student 
could receive values of 0, 1, 2 or 3 for the item) or a value of 1/3 
(so that a student could receive values of 0, 1/3, 2/3 or 1 for the item). 

Hnally, the values assigned to each response must be capable of 
summarization m a single score by means of some mathematical 
formula that clearly expresses the intended meaning. In the Proficiency 
Examination example, a formula meaning ^'percentage of the totid 
content of a discipline that a student has mastered** would be: the sum 
of the values assigned to each of the student's coneict responses in a 
discipline divided by the sum of the vslucs assigned to all correct 
responses m that discipline. Following this formula, if a student 
correctly answered 96 of 120 multiple-choice items with single correct 
responses m a ^ven discipline, his score would be 96/120, or 80%. 
The meaning of this score is: ''The student has mastered 80% of the 
content of this discipline." 

So long as the desired meaning of a score can be stated and that 
meaning supported by the examination content, the method of assigning 
values to correct responses, and the summary formula, it is possible to 
generate scores assessing nearly any skill expected of the students. 

Step 5: set standards appropriate for the decisions to be made 

Since examinations are the principal method of determing the 
adequacy of student perfoimance, each aspect of performance about 
which a decision is to be made must be assigned an appropriate score 



in accordance ^iUi step 4 of the examination plan. These scores provide ~ 
the data from which judgements will be made. In step 5, a stardard of 
acceptable performance is established for each component of the 
examination in order to provide faculty with a basis for a decision 
about whether individual performance is adequate or inadequate. 

There are two principal approaches to the establishment of 
performance standards: normative and criterion. In the former, an 
individual students performance is assessed only in comparison with 
the performance of all others taking the examination. In the latter, 
the adequacy of an individual student's performance is determined in 
relation to an absolute standard established before the examination 
is admmistered 

Whatever the method, application of a standard to a student's score 
results in the assignment of an assessment symbol. For example, based 
on the total scores in a comprehensive examination the faculty might 
wish to award special recognition to the three top students. Here a 
normative standard is employed, ranking performance according to total 
score and gimg special acknowledgement to the first three ranks. In 
another case, the faculty might merely vnsh to determine who "passes" 
and who "fails". Here a criterion standard is employed, and each 
studesfs score is cemparcd ?.ith the minimum acceptable p-crfDrmancs- 
level, resulting in an assessment symbol of P (pass) or P (fail). 

Table 3 illustrates several sets of commonly used assessment symbols 
that might be assigned. to the performance of twenty medical students 
in a single examination. Each set of symbols is further explained 
below. 

Normative standards 

1. Rank: the place an individual score occupies in an ordered listing 
of all scores sequenced according to magnitude. The symbol 1 indicates 
first, 2 indicates second, etc. If more than one person has a particular 
score, all persons with that score are assigned the same rank symbol, 
e.g., 5.5 is assigned to students e and f, who together occupy the fifth 
and sixth rank positions. 

2. Percentile rank: the percentage of students with scores below a 
particular score. The symbol 80 for student d indicates that 80% of 
the students had scores below his. 

3. Z-scores: the number of standard deviation units by which a 
particular score departs from the group average (mean) score. The 
standard deviation is a statistic that describes the variability in a set 
of scores. The z-score of 1.08 indicates that student c is 1.08 standard 
deviations above the group average (the standard deviation in this 
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TABLE a ILLUSTRATIVE ASSESSMENT SYMBOLS 



Normative Criterion 

Student Score 

Rank Percentile Z-score Letter Pass/Fall Letter 



CI 




1 




1.50 


A 


P 


0 


h 


Of 
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1 00 
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68 


18 
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D 
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t 
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F 


F 
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illustration is 9.68, the mean is 74.S, and the student's score is 85). 
Z-scores are useful in comparing a student's relative position in the 
group on a series of different examinations. 

4. Letter-grades: in this case the symbols A, B, C, D, and F indicate 
excellent, very good, average, poor, and unsatisfactory respectively, and 
are based upon percentile ranks so that the categories include 
approximately the following distribution: 10% A, 20% B, 40% C, 
20% D, and 10% R 

Criterion standards 

1. Pass/fail: the assessment symbol P is assigned to aU scores at or 
above the predetermined passing score, and F to those below> For this 
examination, the faculty determined that a score of 69 would be 
required for passing. 

2. Letter-grades: in this case, the faculty agreed in advance that 



42 



ERiC 



scores of 84 and above would be considered outstanding (O), between 
69 and 83 satisfactory (S), and 68 or below unsatisfactory (U). 

The standard selected for judging student performance must be 
consistent with the purposes of the examination. In a Proficiency 
Examination designed 'Ho determine whether any student has 
sufficient mastery of one or more disciplines within the basic science 
curriculum or of the basic medical sciences as a whole," it would be 
inappropriate to apply a normative standard. It is entirely possible that 
nont of the students is proficient enough in all disciplines to merit 
certification. The only sensible standard would be a predetermined 
criterion requirement — a minimum passing score established in 
advance as the minimum acceptable evidence of mastery of the 
examination's content If the standard employed in evaluating student 
scores is not consistent with the purpose of the examination, it is 
impossible for a faculty to make sound decisions. 

Step 6: establish procedures for examination review 

Even the best efforts of skilled examiners do not always yield a 
perfect examination, one that accurately characterizes the abilities of 
the students. Consequently it is necessary to establish procedures to 
evaluate the examination itself before reporting results. Important areas 
for such evaluation include the following. 

Adequacy of content. The examination blueprint should be reviewed 
to determine whether the examination covers the intended content and 
process categories in a manner consistent with the purposes of the 
examination. The examination materials should also be reviewed to 
determine whether or not they meet the specifications set forth in the 
blueprint These analyses are best undertaken before the examination 
is administered and by persons sufficiently familiar with the curriculum 
to make judgements on content adequacy (but not the same persons 
who developed the blueprint and examination materials). Whenever 
deficiences are discovered, the content must be improved before the 
examination is given. 

Clarity of items. It is also important to review individual items for 
conceptual and verbal clarity. This is acxomplished not only by 
pre-examination editorial review but also by statistical item analysis 
after the examination is administered. 

For example, the author of a multiple-choice question may hold 
the following opinion of the 5 possible responses: 
option 1 — incorrect, easily recognized as such; 
option 2 — correct; 

option 3 — incorrect, easily recognized as such; 
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option 4 — incorrect, diflScuIt to distinguish from correct; and 
option 5 — incorrect, somewhat difficult to distinguish from correct. 
From this description it would be expected that very few students 
would select options 1 and 3, that a larger proportion would select 
option 5, and that even more would select options 2 (the correct option) 
and 4; and actual results might than take the form shown in line A 61 
Table 4. If, however, the results shown in line B were obtained, the 
author might suspect either that the studepts do not know the content 
or that there is some deficiency — seme ambiguity — in the item itidf. 
Any items found to be ambiguous by this post*examination review 
should be deleted from the examination before final scores and assess- 
ment symbols are assigned. 

TABLE 4. ITEM ANALYSIS 



Option 1 2 3 4 5 



A. V9 students 2 58 4 25 11 

B. Vo students 2 31 4 25 38 



Consistency of results. Two general methods of assessing reliability 
(consistency) may be employed: repeated ^dm nistration of the examina- 
tion to see whether it assigns similar scores to each student each time 
(test— retest methods); and comparison of parts within the examination 
to see whether scores assigned to a student by one part are similar to 
scores assigned by other parts (internal consistency methods). For 
example, an examination with low reliability as determined by internal 
consistency methods might have several items that many of the better 
students (with hi^ total scores) missed and the poorer students (with 
low total scores) answered correctly. Such an examination would be 
improved by deletion of these items before final scores are computed 
for the students. 

Because it is important that both faculty and students can have 
confidence in the decisions made on the basis of an examination, it 
is essential that every examination be adequately evaluated Any 
examination that fails to meet technical standards of adequacy cannot 
serve as a rational basis for decisions about student performance. 

Step 7: report the results 

The final step is that of reporting the results to all persons involved in 
the decisions being made on the basis of that examination. Exactly 
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what is reported and to vrhom depends upon the purposes for which 
the examination was administered. 

For example, the results of a diagnostic examination aimed at 
assessing a student's mastery of five content areas and his ability to 
perform three cognitive processes would be sent to the student and to 
his faculty adviser, showing a separate score and/or assessment symbol 
for each content area and for each cognitive process. This set of scores 
would then enable the student and his adviser to pinpoint specific 
weaknesses that warrant special attention. Because of the diagnostic 
nature of the examination, no report would be entered on the student's 
permanent record 

In the case of a certifying examination it may only be necessary to 
report the student's total score and its associated assessment symbol, 
since here the purpose is usually to record overall competence and not 
the specific components of competence. This information would be 
reported to the student and to the faculty committee responsible for 
certification and become part of the student's permanent record. 

In any case, the contents of the reports and persons who receive 
them must be consistent with the purposes of the examination, to ensure 
that all relevant information is reported appropriately to all (and only 
those) persons entitled to knowledge of it. - - - • * 



ADDUIONAL READING 

This paper outlines the basic issues involved in using examination 
data for decision-making but does not provide detailed explanations of 
the skills necessary for carrying out an examination plan. The following 
texts aie recommended to those who wish to learn more. 

Bloom, B. S. et al. (1971) Handbook on formative and summative evaluation of 
student learnings New York, McGraw Hill 

Ebcl, R. L. (1965) Measuring educational achievement^ Englcwood Cliffs, N.J., 
Prentice-Hall 

Furst, E. J. (1958) Constructing evaluation instruments. New York, McKay 
Miller, G. E. et al. (1962) Teaching and learning in medical school, Cambridge, 

Mass., Harvard University Press 
Thomdlke, R. L., ed. 1971) Educational measurement, 2nd ed., Washington, D.C., 

American Council on Education 
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DYNAMICS OF A LEARNING GROUP 



Shirley Perlmutter* 



The knowle/Jge explosion of the last 25 years has vastly increased 
the subject-matter to be taught in today's medical schools, and the 
demand for trained physicians has also accelerated. Medical schools 
are often understaffed and overextended, yet faculties must face students 
who have become increasingly vocal in demanding that classroom 
content be more relev nt to the skills needed on the job. The medical 
school teacher &f t6day finds himself facing "a dilemma verj''much like 
that of the sorcer«*r'G apprentice, whose every effort to stem the tide 
merely sent the waters higher. The purpose of this paper is to propose 
one way by which health professions teachers may be released from the 
demand that seems to be imposed by the imending growth of knowledge 
and heightened student expectTitions. To attain this goal, some of the 
assumptions underlying traJitional teacher practices must first be 
examined in order to explore alternative approaches. 



TRADmONAL TEACHER BEHAVIOUR 

Traditionally, the behaviour of teachers in medical schools has been 
characterized by a high degree of control over and direction of students, 
a posture presumably based on the assumptions that (1) if left to their 
own devices students would be apathetic about or resistant to the task 
of learning; (2) students do not know what they need in order to 
achieve effective professional performance goals; and (3) the teacher 
has the necessary expertise to determine the nature and scope of the 
subject-matter to be mastered. Thus it is commonly regarded as a 
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faculty responsibility to motivate and direct students, to control their 
actions and influence their behaviour in such ways that the educational 
goals will be achieved. Teachers see themselves, and are seen by their 
students, as the experts who will present material and determine the 
extent to which students have mastered it With the pressure of time 
and a constantly expanding factual content, teachers are rarely 
encouraged, or provided with the opportunity, to slow down and 
examine more closely the implications of these teaching practices for 
learning. 

While this traditional approach to teaching is undoubtedly an 
efficient means of transmitting large quantities of information to large 
numbers of students, there is a real question of whether they grasp 
what is presented. It is also clear that such an instructional format 
produces students who are dependent upon experts for direction rather 
than independent in the pursuit of learning that must be lifelong. They 
come to believe that all expertise does reside in tiieir teachers, and that 
performance should be judged by the expert standards that teachers 
alone establish. It should come as no surprise, then, that students are 
particularly concerned with how their behaviour is perceived by 
teachers, and express urgent needs to be directed, assigned specific 
tasks, *and told that they are carrying out tfiose tasks in the proper 
way. When deprived of these accustomed external sources of support 
and direction it is also understandable that they may experience 
considerable anxiety, if not frank hostility. And all this occurs in an 
atmosphere that is regularly described as unpersonal and lacking in the 
important element of close interaction sought by both students and 
teachers. 



A NEW FORMAT 

As it becomes increasingly apparent that students can no longer be 
taught all they will need to learn, the focus of education must shift 
to teaching students how to learn and how to be independent, 
self-directed, and equipped to deal with an ever-changing environment 
It has been said that if a student can be trained to be an effective and 
independent learner he need not be filled with all the information he 
can contain before going into a new job. He will have tlie capacity to 
learn whatever he needs to learn. Indeed, he will have to generate that 
capacity in any case, whether he is trained to do so or not, since no 
classroom can prepare him for every contingency, for every aspect of 
life and work encountered outside the classroom. 

Training students to become self-directed learners implies a set of 
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postulates about students and the learning process different from those 
described earlier. It assumes: (1) that each student enters the learning 
situation with an inherent desire to learn and a capacity to accept 
responsibility for his own learning; (2) that he has untapped 
resources that he may utilize to identify his own learning needs, 
assess potential learning resources available in his environment, and 
participate with teachers in determining the content to be mastered; 
(3) that significant and rapid learning will occur when he perceives the 
subject-matter as relevant to his purposes; and (4) that encouraging 
students to confront in an educational setting the kinds of practical 
problems they must foce in the real world is one of the most 
effective modes of promoting learning. 

One place in which to foster such independent learning is the small 
group. The idea of reducing class size to improve the quality of 
learning is not new. It has often been proposed in the hope that 
smaller numbers will allow more opportunity for student— teacher 
interaction, for the quiet and shy student to participate more freely, and 
for all students to become more actively involved with the content. 
But the expectation that a reduction in class size will bring about these 
changes is unrealistic. A one-way flow of communication from teacher 
to students can occur in a sm'all group as well as a large one"— and 
often does. If the potential of the small group (10 — 20 students) is to be 
realized fully, then traditional teaching behaviour must be modified to 
match the assumptions about students and learning listed above. In 
order to accomplish successfully this different teaching style, a teacher 
must be knowledgeable about the dimensions of the group process, the 
developmental sta^s of groups, and how to conduct himself in a 
manner that supports group devdopment and the achievement of small 
group goals. 

DIMENSIONS OF THE GROUP PROCESS 

In order to avoid any possible misunderstanding it should be noted 
at once that group discussion is not intended to bf; random but 
rather to deal with some identified content But whatever the content, 
all groups develop an operational process — a style of working together 
that has great impact on the quality of the product — and a set of 
relationships — patterns of interaction among the participants that affect 
the character of the experience. Whether the teacher recognizes them or 
not, these factors are at work, either facilitating or impeding the 
achievement of group goals. It is the teacher's task to be sure that the 
former, rather than the latter, is the dominant mode. If he is sensitive to 
this process he \M give particular attention to (1) the amount of 
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individual participation; (2) iiow members try to influence each other; 
(3) how group decisions are made; (4) how subgroupings are formed; 
and (5) how feelings are expressed. 

In all groups, regardless of the purpose for whi^rh they have been 
formed or the nature of their participants, the proceis evolves through 
six general stages: 

(1) Stagp I, getting acquainted, is characterized by polite and 
superficial interaction between the members. Even when they have 
met before, or even worked together, the formation of a new group 
creates an imfamiliar settmg in which members must get to know 
each other anew. 

(2) During stage 2, What am I here for?, members experience a 
period of ambiguity, a lack of direction or purpose. They become 
frustrated, angry, confused, and impatient 

(3) Out of this sense of disorder stage 3, a bid for power and 
inftuencCy emerges. Members begin to attempt to create order out of 
what has become intolerable confusion. It is at this time that working 
rules for the group begin to be established and commitment to 
membership begins. 

(4) In stage 4, What can I do Jo help?y group cohesion appears. , 
Natural resources and leaders are identified and members negotiate 
around who will do whaty and when to achieve the group task. 

(5) Stage S is characterized by a sense of accomplishment. Through 
the negotiations that occurred during the stage 4 members are able to 
collaboratey to execute and complete the task. 

If a group is formed for the purpose of accomplishing more than one 
task, each time a task is completed and another proposed the group will 
recycle through the first five stages. While the stage of getting acquainted 
may be less obvious the second time roimd, even this does occur. A new 
task usually creates the need for different resources and behaviours that 
were not required by the previous task. Thus as new tasks arise 
members must gpt to know each other in new ways. 

(6) Finally stage 6, terminationy occurs when the members recognize 
that at a specific time the group will come to an end and the participants 
will disperse. During this last stage (such as the final week of a 4-week 
workshop or the last month of a school year) members be^ to dis- 
engage from the group and to reduce their involvement with and 
commitment to group issues. Some time may be spent in this stage 
reviewing events during the life of the group, in an attempt to 
summarize and put a closure to the experience. 

One of the functions served by the model just outlined is that of 
providing a perspective for certain cyclic phenomena inherent in group 
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work. For example, a let-<lown feeling and sense of disorganization 
often follow the "high" feelings associated with the sense of 
accomplishment in stage 5, This reappearance of "disorganization" is 
frequently interpreted as indicating poor group functioning, when it is 
in fact a natural phenomenon that reciurs during the life of any group. 
The model also underlines the sequential nature of the stages and the 
importance of each. Even the period of ambiguity and chaos in stage 2 
is productive. K members can tolerate some degrse of frustration and 
uncertainQr and refrain from prematurely organizing the group (stage 3), 
opportunities for potentially innovative and creative problem-solving 
around tasks will be enhanced. 



CONDUCTING AN EFFECTIVE GROUP 

In such small groups the ideal teacher is one who more often listens 
than talks, questions rather than answers, moderates rather than 
dictates, and stimulates others rather than performs himself. In such a 
setting the personal conmiitinent to learning becomes overt and 
shared; both students arfd terxher can ask for fe&dback; personalized 
two-way communication can develop among students, and between 
students and teacher, and evaluation can be immediate, ongoing, and 
multidimensional from self, peen, and teacher. 
' In order to help readers to gain some concrete notion of how these 
principle? can be translated into practice, a sequence of events is 
described below, illustrating how a teacher might facilitate the 
development of a climate that encourages self-dkected and independent 
learning in the small group setting: 

(1) The teacher opens with a description of his goals by presenting an 
overview of the objectives, the content to be mastered, and a proposed 
schedule of activities. Additional stafi resources may also be introduced 
or otherwise identified. 

(2) After this general introduction students are asked to describe 
their goals. These expectations are explored together through a problem 
census depicting individual learning needs. 

• (3) Teacher and students then compare the general programme 
objectives with the individual learning needs, noting similarities as well 
as differences. 

(4) Learmng needs identified by both teachers and students are 
translated into specific objectives and listed in order of priority. 

(5) Teacher and students then identify jointly the objectives they can 
reasonably expect to achieve during the programme. 
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It is important that these five activities occur at the outset, in order to 
establish as a norm the active dialogue among sudents themselves as 
well as between students and faculty. The five steps assume that while 
the faculty has a set of general objectives and an instructional strategy 
to implement their achievement, there is suflficient flexibility in the 
programme to allow responsiveness to student input This docs not 
imply that students determine what will be taught but that the oppor- 
tunity exists for shifting emphasis from one topic to another, for the 
indusion of a related issue not originally planned, or for the elimination 
of something already learned, for example. It is also worth notmg that 
the five steps correspond to the five stages of group development The 
staff introduction and statement of goals vis-d-vis the student expres- 
sions of individual learning needs and expectations is a way for faculty 
and students to begin to get acquainted (I). The expression of individual 
learning interests and needs is often elaborate as well as diffuse, so 
that students and teachers alike may experience a sense of confusion 
(II) and of being overwheUned (How will all this get done?). Com- 
paring student and teacher goals, listing them according to perceived 
importance, is a way of creating order QH) out of the confusion. 
Negotiating (IV) a joint statement of ^oals also involves ncccssj^ry 
collaboration (V). 

This sequence, which generally takes 2-3 hours, illustrates the basic 
steps required to build a cooperative learning group. It may seem 
excessive in a class that meets for only one hour three times a week 
for a term, but the time invested in this way at the bc^nning will pay 
handsome dividends in the rapidity and solidity of subsequent Icammg. 

But not all small groups enjoy the luxury of an extended period 
during which participants may work together; some may meet only once, 
and then for a short period. Yet even in the course of a single 
hour a skilful teacher may put into practice some of the principles that 
have been cited as critical contributions to productive group work and 
to the long-range goal of encouraging self-sustained independent 
learning. An illustration of such an effoit, one that was created and 
tested by a participant in a recent teacher-training programme, is set 
forth in the accompanying table. 

A teacher's primary concern, when he is trying to adapt small-group 
learning methodology to settings where the content is relatively fixed, 
is to devise a means of actively engaging students in the learning 
process. Even ^en time and content do not permit the group to evolve 
through all five stages, the model may be useful in identifying ways in 
which the learning activities may be designed to elicit maximum student 
participation. 

For example, in the biodegradation exercise (sec the accompanying 
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BtODEQRADATtON — SMALL-GROUP EXERCISE* 



ObitcUm: At the conclusion of the 1-hour session students will be 
able to: 

(1) identify the reasons for modem sewage disposal; 

(2) predict the effect on the ecology of a large lake of the 
release of a large volume of untreated or partially 
treated sewage; 

(3) predict the possible effect on the health of a com- 
munity of the release of a large volume of untreated or 
partially treated sewage In nearby water courses; 

(4) evaluate data on the biochemical oxygen demand of a 
specified sample of water. 

Detlgn: (1) Introduction. 

Teacher introduces the topic and provides an overview 
10 min of the hour. 

(2) Class breaks Into groups of 3. 

Handouts distributed on Lake Erie (**Death of a Lake*') 
and on gastrointestinal outbreaks. 
15 m;n • Taak -!*ft the dist?ases spread via sewage . 

-understand the concept of biochemical oxygen 
demand. 

(3) Return to full group — one member of each trio sum- 
marizes the deliberations under 2 above. 

20 min (4) Short lecture, illustrated with visual aids, followed by a 
discussion of sewage treatment. 

(5) Class forms two groups of 6. 

Handouts distributed on biochemical oxygen demand. 
10 min Task -to understand tests and interpret their signifi- 
cance. 

(6) Return to full group. 

Final summary and distribution of a synopsis of the 
session. 



• Reproduced by permission of Dr Adrian Lee. Department of Medicot Microbiology. 
Faculty of l^adiclnft. University of New South Wales. Sydney. Australia. 

figure) the teacher began with stage two. When he had identified himself 
as the authority with the intention of directing students to a specific 
topic, stage 2 was not an issue for the full group or the smaller 
groupings. The i^al group work was initiated with stage 3, when the 
class yas divided into trios, each of which was required to designate 
a reporter. Stages 1 and 2 then occurred briefly as members began to 
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know each other through the ways in which each responded to the task 
in hand, and as they sought to ascertain whether there was any residual 
confusion about the task. But the emphasis in stage 3 was on identifying 
resources within the group, so that they could progress through stages 
4 and 5 to completion of the task. This sequence would then be 
repeated when groups of six were formed for the second task* Stage 6, 
termination, occurred when the groups reassembled as a full class for 
the final summary and distribution of a synopsis. But at every stage the 
focus was upon the students and their responsibility for learning, not 
upon the teacher and his opportunity to instruct. 



CONCLUSION 

Self-direction and independent learning are important general 
objectives in any programme of medical education, but this does not 
imply that even the most vigorous proponents of this stance advocate 
anarchy, in which every student does what he wants to do, any more 
than it is tolerable for every faculty member to do only what he wants 
to do without regard for others. There is a place for lectures and drill, 
for student listening as well as fsculty taSring. But if individu^d karoing. 
is to be optimized, faculty members must be prepared to exploit the 
principles outlined here, whether they are working with a few or with 
many students. It is, of course, more difficult with large numbers, but the 
teacher who accepts the fundamental validity of the assumptions upon 
which such an approach is based will find ways to incorporate those 
principles wherever he teaches. Subgrouping a large class to allow 
smaller numbers to engage even briefly in an active process of learning 
may seem a waste of precious time — but only if the teacher's role is 
seen as information source rather than learning facilitator. To the extent 
that they encourage students to find in their peers some of the stimulus 
and guidance that in conventional instruction depend upon teachers 
alone, they create an ever-widening group that can continue to provide 
mutual support in the act of learning independently and outside the 
classroom. And that, of course, is where it must occur for the rest of a 
professional career. 
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TEACHING LARGE GROUPS 

George E. Miller^ 



As higher education throughout the developed world has evolved 
from the privileged experience of a small elite to the reasonable 
expectation of a substantial segment of contemporary youth, a growing 
volume of complaints are heard about decline in quali^ of that 
experience as the mmiber of students increases. Today, in developed 
and developing countries alike, those responsible for health professions 
education programmes are confronted with the problem that the social 
demand for trained professionals has dictated a substantial increase in 
student numbers in the absence of a proportionate expansion in what 
are believed to represent minimal resources of personnel and facilities. 
There is no faculty dissent from the demonstrable fact that large 
numbers of students can be processed throu^ these educational 
programmes; their concern arises from the apparent impediment large 
numbers pose to producing graduates who have acquired not only the 
necessary body of information about health and disease but also the 
capadty to apply that information successfully in the management of 
the medical problems they must ultimately face, and the attitudes of 
personal responsibility for continuing their own self-education if the 
socially unacceptable alternative of professional obsolescence is to 
be avoided. 

The issue is how to resolve the educational problem created by a 
discrepancy between large numbers of students and limited resources 
with which to train them. Rational analysis suggests three possible 
solutions: 

1 . Reducing the number of students is probably not a viable approach- 
It would be difficult to mount any convincing argument that large 



^ Director, Center for Edacational Development, Unlvenlty of IHinols Cbttete of Medldne. 
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numbers decrease educational quality, for the limited evidence available 
suggests that class size is not the critical variable in the quality of 
individual learning.^* ^ In addition there is still to^ much to be 
accon^Iished in improving man's health, too much faith that this cannot 
be dcme without a substantial increase in the number of practitioners, 
and too much prestige associated with the physician role, to expect that 
society will reverse a trend that has won such widespread support 

2. Increasing the resources seems an equally unproductive path to 
pursue. This does not imply that medical schools have all the resources 
they need or that faculties should relax their efforts to acquire more. 
But the economic fact is inescapable that medical education is already 
among the most expensive of university programmes and there are 
odier important claims on that portion of the gross national product 
of any nation that can legitimately be assigned to education or to health. 
There mil certainly be continuing discussion of the issue of funding 
but it is unrealistic to expect an early solution to the problem of 
numbers as a result of a substantial and widespread increase in 
resources. 

3. Using the available resources differently. Since the major resource 
in any educational enterprise is the foculty, whose members are surely 
capable of learning how to use themselves differentiy, this approach 
would appear to have the greatest immediate and long-term potential 
utility. The areas of teaching staff change that mi^t be most firuitfully 
explored are: (a) altered perception of the teaching role; (6) the employ- 
ment of different instructional strategies and tactics; and (c) the 
development of new instructional skills. 



THE TEACHER ROLE 

Medical teachers have generally won teaching appointments only 
after long study and the achievement of recognition as biomedicd 
scientists or medical practitioners. Under these circumstances it is 
understandable that so many teachers should see thdr principal function 
as that of sharing with students the knowledge they hav^ acquired, 
the skills they have developed, and the wisdom they have won. The 
teacher role thus becomes one of telling and showing, and the student 
role that of listening and watching until the teacher's learning has 
been transmitted. 



^ us Govenuncftt Printlns OfBce (1960) Efftctlveness In teaching. New dimensions in higher 
education. No. 2, Washington, D.C. 
* Sanazaro, P. J. (1966) Oass size in medical school. /. mtd. Educ., 41, 1017. 
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Such a rote may be appropriate in a static field where final (or at least 
semi-final) truth has been established and the student task is simply one 
of absorbing fixed patterns of behaviour. It is surely not appropriate 
in a rapidly developing field where "facts" are always changing, new 
professional skills are constantly emerging, and the student's task is that 
of learning how to weigh and judge both the new and the old throughout 
his professional career. Here the teacher's responsibility is not to 
transfer his knowledge to students but to join them in a never-ending 
search for knowledge. In some instances what he has learned may prove 
useful to them, m others their challenges may prove useful to him, but 
in either instance the exchange is in both directions, not merely one way 
from authority to apprentice. 

Many teachers who are intellectually prepared to accept such a role 
find it difficult to translate into practice. If they are not to engage 
primarily in showing and telling, what are they to do? Three major 
tasks can be clearly identified: 

1. Helping students to define the cbjective of a learning experience. 
Anyone who has tried to ''assist" a neophyte to learn anything knows 
how frustrating the work can be for both parties. The teacher who loses 
patience or fails to understand the purpose of the effort may respond 
too quickly when students cry, ''just tell me what to learn and I will 
learn it" Only as a teacher comes to accept as his primary role that of 
helping students to learn how to learn, rather than to learn what he 
knows, will the interaction become both fruitful and satisfying. Once a 
student clearly understands the educational objectives in terms of his 
own career goals, his learning will generally proceed at a pace that 
astonishes both himself and his instructor. 

"People learn what they want to learn" is an old aphorism too often 
overlooked by teachers who forget that their most important function 
is tiiat of helping students to discover a need to know (i.e., the 
objective) — and then getting out of thdr way. 

2. Helping students to identify the available learning resources. 
Among these resources are certainly individual teachers, but if there 
is honest acceptance of the principle that all knowledge does not reside 
in one teacher — or one department — then students will be helped to 
discover and use the many others that can be found on all sides. The 
most generally available resources are the printed materials collected 
in libraries, now increasingly supplemented by a growing array of oth^ 
learning aids such as audiotapes, slides, films, videotapes, and 
programmed devices. 

Ihe problem is really not one of availability (although in some parts 
of the world the supply of books and journals may be limited) but of 
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acceptance. Teachers who see their major teaching function as that of 
transmitting information often find it difScuIt to identify a source as 
accurate, as current, or as complete as their own knowledge, and thus 
are unable or unwilling even to reconunend a textbook — unless they 
have written it More consistent with the role highlighted here is the 
attitude of teachers who recognize the ever-changing nature of knowl* 
edge and are comfortable with having students learn things they may 
not personally accept — even things they may think wrong — provided 
that at the same time those students also learn how to make judgements 
and to defend with logic and reason the conclusions they have drawn. 
Such teachers will help students to see, appreciate, and use the infinitely 
varied learning resources a medical school provides, not merely the 
faculty member who stands before them. 

3. Providing students with non-}udgemental feedback on the extent 
to which they are achieving the learning objectives that have been 
agreed upoa The keyword in this phrase is *'non-judgemental." To be 
sure, it is essential for quality control of education to make a judge- 
ment at some point of the extent of the individual student*s learning, 
but the best evidence now available suggests that any student who 
has the intellectual ability required for admission to medical school 
is able to master its programme of study if given appropriate guidance 
and adequate time. The most important guidance function a teacher 
can fulfil is that of regular and systematic determination with each 
student of progress towards educational goals — a joint appraisal of 
what has been learned and what has yet to be learned, so that sub- 
sequent study can be directed towards filling the remaining gaps 
rather than further refining what has already been achieved. If such 
periodic, non-judgemental feedback is accomplished effectively, a 
student can face the ultimate judgemental assessment confident that 
he is abe to meet the minimum performance requirements. 

Assumption of the teaching role described here and employment of 
the ir^tructional strategies enumerated may demand a fundamental 
change in the attitudes of many teachers. Not least, there must be a 
willingness to accept students as responsible individuals who are able 
to learn independently and who need support more than instruction. 



NEW INSTRUCTIONAL TACTICS 



It is almost inevitable that, when encoimtered for the first time, the 
views set forth above will be interpreted as an attack on the lecture 



method of instnictioiL The removal of emphasis from "teaching*' (in the 
sense of telling) and the placing of emphasis upon individud and 
independent learning certainly suggest such a conclusion. But it is 
important to draw a sharp distinction between the lecture method of 
instruction^ and the lecture system of education* It is the latter, not the 
toxmeTf that must be discarded if the broad educational goals set 
forth in the opening paragraphs are to be achieved A teacher who 
abandons the role of primary information source and accepts the role 
ot learning facilitator may still use lectures but will use them sparingly 
and in a manner that serves goals more general than the transfer of 
information. 

At the same time it must be acknowledged that in some parts of the 
world printed or audiovisual instructional materials in the native tongue 
of most students are virtually nonexistent Here the alternatives seem 
to be: (a) the use of lectures or (6) the provision of printed information 
in a second language. Since books are often beyond the means of 
individual students the lecture system may thus by default become the 
dominant instructional mode, but such a system is attended by very 
serious hazards. Quite apart from the dependent learning habits that it 
encom^ges, anyone who has read carefully the literature on lectures wiU 
be appalled by the inacciuracies that occur as the lecturer's notes become 
the student's notes and by the serious limitation on information 
absorption that occurs after a relatively short time (15—30 minutes). 
Under these circumstances it is probably wiser for teachers to record 
their words for transcription in the native tongue or for playback by 
individual students or groups. Thb will at least introduce some measure 
of flexibility into the programme and an opportunity to use for other 
mstructional purposes in later years the time that might be invested 
in repeating die same lectures. Despite the feeling teachers often have 
that lectures must be updated each year, the fundamental concepts to 
which such instruction for medical students should be addressed rarely 
change so rapidly. 

But whether lectures are used as the principal method of instruction 
or merely as one element of a more varied instructional programme, 
the method can probably be used far more effectively than it is in most 
classrooms today. A change in dominant lecture style is especially 
important when the student body is large and opportunities for 
individual encounters with teachers are limited. A skilful lecturer can 
convey even to an audience of 500 some sense of personal interaction 
if he will avoid the impulse to tell all he knows, interrupt his discourse 
with concrete and relevant iUustrations, and challenge his audience 
periodically to apply what has gone before or anticipate what is to 
come. The details of how to modify and improve lectures have been 
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described elsewhere*; the purpose here is merely to remind readers that 
even this most widely used instructional method can be employed to 
serve more general objectives that are particularly significant if the 
problems associated with large classes are to be met successfully. 

There are other methodolo^cal tactics that require no additional 
resources, merely a willingness to accept the new role ahready outlined 
A simple illustration is the distribution of a clear and concise statement 
of learning objectives for a programme or course before instruction 
be^ns. Such a statement must be more meaningful than, for example, 
"to appreciate the importance of tranquillizers in the treatment of 
emotional disturbances". It must be couched in terms of what a student 
must do to ensure that both he and his teachers know that he has 
gained such appreciation. The methods of deriving and stating educa- 
tional objectives have also been dealt with elsewhere^; it is the use to 
which they are put as instructional tools that is of concern here. Too 
often students are placed in the uncomfortable and frustrating position 
of having to guess the objective of an educational offering and waste 
untold energy in trying to anticipate what a teacher really wants them 
to learn (not to mention for what purpose). When the objective is both 
specified and shared in advance many students find ways to accomplish 
that learning in a manner more closely matched to a personal learning 
style than the faculty-designed method, and often in significantly less 
time. In the case of large groups, making such provision for individual 
differences is particularly important 

If it is also possible to provide a wide array of aids to independent 
study, so much the better. First among these are printed materials, 
available in such rich variety in so many parts of the world. More 
sophisticated instructional technology such as single concept films, 
audio-cassettes, programmed instructional devices, and simulations may 
have greater appeal but what is important is not the device itself but 
how that tool is used. Too often these aids to learning are regarded as 
add-ons, mere supplements to the conventional source of wisdom, the 
"teacher". If a new sense oC oersonal responsibility for individual 
learning is ever to be achievt.d by health professions students, these 
instruments must be integral elements of a flexible and individualized 
programme, not merely extras to be enjoyed if time permits. 

Among the most readily available, yet least used, aids to such an 
educational enterprise are the students themselves as teachers, Staff 



1 Bughman, E. (1973) The lecture method of instruction, hi: Development of educational 
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may not be prq)ared to acknowledge it, but the evidence is persuasive 
that a substantial amount of the most persistent learning that occurs in 
any educational programme results from peer interaction. Rather than 
allow this to take place in unplanned and random fashion, teachers 
who are willing to accept the primary role of guide might harness this 
vast potential to serve defined educational objectives. Encouraging the 
more advanced studrats to help the less advanced, the more senior to 
tutor the more junior, not only multiplies the instructional resources of 
an institution but also provides those who teach with an unparalleled 
opportunity to further their own learning. A fifth-year medical student 
may not be able to provide for a second-year student the depth of 
instruction in biochemistry that a biochemist woidd oSer, but the 
communication is more likely to match the learning readiness of the 
novice. It will at the same time crystallize his own understanding of the 
topic, for in the most succesful teaching/learning exchange the teacher 
profits quite as much as the taught Such factors should encourage the 
utilization of students as teachers in any setting; where classes are large, 
the further advantage of reducing the size of learning groups, through 
increasing the number of teachers, is also achieved. 

While peers may provide the most readily accessible pool of 
additional teaching talent there are others, rarely used, from whom 
health professionals can also learn: the practitioners of these professions. 
So long as the teaching role is regarded principally as that of informa- 
tion source it is understandable that academics at the frontiers of a 
discipline are reluctant to entrust students to practitioners who are not 
always 'Hip to date.** But if the teacher becomes a guide, one who helps 
a student to identify what he needs to learn and who provides oppor- 
timities through which that learning can occur, then practitioners can 
become important contributors to this effort What they have to offer 
may also be considerably closer to the career for which most students 
are preparing than the contributions of academic specialists, who now 
play the dominant role in most programmes of medical education. 

Finally, if such non-traditional activities are to further the most 
effective learning and the most efficient use of learning resources there 
must be some ongoing mechanism for students and faculty to assure 
themselves that learning is being accomplished. It is here that the 
diagnostic examination fulfils such an important function. Students 
everywhere have been conditioned to perceive examinations as a threat 
to their academic survival, and omfortunately too many teachers still 
use examinations in a manner that justifies such a conclusion. It may 
take time, and will certainly require that teachers behave more in 
keeping with the role of facilitator rather than judge, but with these 
provisos there is no reason to doubt that students can be brought to 
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the point of seeking out diagnostic examinations to help them define 
more precisely both what they have learned and what they have yet to 
master. The mechanisms for implementing such a programme have 
been dealt with at length elsewhere^ and need not be repeated here. 
But it may be important to emphasize that accomplishing the aim of 
increasing individualization of learning opportunities for large classes 
is unlikely unless there is some provision for regular, personal, non- 
judgemental feedback of learning achievements and remaining de- 
ficiencies. 



NEW SKOXS 

Even teachers who are persuaded that they must accept a new role 
and are willmg to try new instructional methods will probably discover 
that their preparation has not provided the necessary skills for con- 
ducting educational programmes so different from those they have 
known. In fact, there is little hope th;it what has been outlined here can 
be achieved unless teachers are willhig to develop new pedagogic skills 
and further refine some they may already have acquired (e.g., devel- 
oping greater proficiency in deli>ering lectures). 

Among the most vital is the skill of educational planning. It is 
probably no exaggeration to say that most medical teachers now 
develop an educational plan on the basis of subject-matter content 
Either through personal experience with the field in which they are 
expert or through the guidance provided by standard textbooks, they 
decide what must be taught, and in what sequence. But the goal of 
education is leammg, not teaching; and the distinction is more than 
semantic: learning implies a change in behaviour, not merely exposure 
to instruction. Thus educationalplanning must begin with clear delin- 
eation of what students should be able to do at the conclusion of a course 
of study, after which the knowledge they need can be more rationally 
defined. Since the primary purpose of health professions education is to 
produce practitioners who are able to promote the health of society, 
this definition will relate directly to the health of the community they 
are to serve. The derivation and specification of objectives, and the 
mechanism for their incorporation into educational programmes, has 
also been described elsewhere^* ^; it is to these descriptions that teachers 
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ready to acquire new skills for dealing with large classes may wish to 
turn for further guidance. 

Obviously, if individualized instruction is to be made available to a 
large body of students it will be necessary to produce or to select many 
new aids to learning. The means of accompUslung these goals have also 
been described.^ It is important to insert a cautionary reminder here. 
The purpose is not merely to develop new materials that exploit the 
mechanical potential of a growing array of simple and complex devices, 
it is to facilitate learning. Tools have no value in and of themselves. 
If they are misused their potential to impede learning may be realized 
Thus in this arena the skill that teachers must acquire is not only one of 
developing suitable materials to aid learning, but also of using them 
appropriately as an integral element of a total learning system. 

Most important, however, may be the ability to provide individual 
students with the encouragement, guidaace, and support they need in 
order to learn most successfully. Although such skills are important in 
any educational environment they become critical elements in a 
programme that must accommodate large numbers of students and 
where the opportunities for individual or small-group exchanges 
between teacher and student are so limited that each encounter must be 
maximally effective. 

The techniques for organizing and moderating small-group discussion 
have ahready been dealt with.^ Rather than repeat here what has 
been said more fully there, only three items will be mentioned 
that seem to be central to the dialogue which, to facilitate learning, 
must encourage students to disclose their ignorance, assist them to 
discover a need to know new things, and acknowledge the achievements 
that they demonstrate. In order to accomplish this a teacher must 
develop a high level of skill in questioning thoughtfully, listening 
perceptively, and responding supportively. 

It may seem strange that the first of these skills is questioning, since 
Quizzing is already extensively used in educational programmes for the 
health professions. It is important, however, to focus upon the difference 
between interrogation, which is certainly widespread, and inquiring^ 
which is far less commonly practised. One takes the form of a 
catechism, where question and answer match a predetermined pattern 
and substance; the other takes the fonn of exploration, in which 
question and answer are the beginnmg of a search, not its end. In the 
former a teacher sets the questions and the student is expected to 
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provide correct responses, while in the latter a teacher may set the stage 
for questions but does not set the limits. The interrogator is interested 
primarily in assessing the knowledge a student has acquired; the 
inquirer is more concerned with identifying how he can be helpful in 
furthering a student's learning. Where the former, following a specific 
reading assignment on perforated peptic ulcer, might ask "What are 
the signs and symptoms of...?", the latter would ask "What still puzzles 
you about...?''. The interrogator seeks &cts, the inquirer looks for 
reasons. 

The second critical element is the skill of listening. It must be 
surprising to any objective analyst of formal education to discover how 
completely the process is dominated by teachers talking, as though 
listeniag were equivalent to learning. Commimication is the keystone 
upon which the educational arch is built, but to be effective the flow 
must be in two dkections. This requires teachers not only to be quiet at 
more than infrequent intervals but also, by thoughtful listening and 
inquiring response, to encourage students to question and challenge as 
fully as they are being questioned and challenged. The skilful listener 
is also sensitive to messages commimlcated by more than words — to 
the non-verbal expressions that may signal anxiety, hostility, lack of 
interest, or preoccupation with other things — and will use these data, 
as well as the words, in fashioning a response. Students may prefer to 
avoid what is not verbalized, but the purpose of the interaction is not 
to do what they want but to do what is necessary to facilitate their 
learning. 

The supportive skill thus comes into focus. Support is provided by 
encouragement and approbation, which should be given sincerely and 
as often as is justified. But support is also provided by revelation and 
critique offered with kindness and without judgement Few students 
wruit their deficiencies highlighted and it is particularly regrettable that 
few faculty members are skilled in accomplishing this necessary task 
in a manner that is perceived as helpful rather than censorious. To 
provide critique without conveying negative criticism is the goal, for the 
one fosters further learning while the other merely arouses a defence 
reaction. Until a teacher acquires this skill his effectiveness in the role 
of educational counsellor and guide will always be limited. 



CONCXUSION 

The problem of large class size, which is encountered in so many 
medical schools in both developed and developing countries, is unlikely 
to vanish in the foreseeable future and the associated educational 
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problems are unlikely to be resolved by the massive infusion of new 
resources. The most realistic approach to the question seems to be that 
of using differently the resources now available, and particularly the 
most critical of these — the teacher. 

While this paper has drawn attention to a different role, alternative 
educational strategies, and refined pedagogic skills that individual 
teachers can employ to mcrease thdr effectiveness in dealing with large 
numbers of students in a more individualized manner, no concrete 
proposal for altering curriculum organization has been made, since 
structure seems less important than the process by which learning is 

FACULTY MAN-HOURS REQUIRED FOR A aO-WEEK BASIC SCIENCE COURSE 
FOR 300 STUDENTS 

Conventional organlution Non*trtdltion«t organtzatlon 
S«nlor staff Assistants Senior staff Assistants 



Lecture (full class) 
delivery 
preparation 

Laboratory* 

delivery 
preparation 

Independent study^ 
(Identification and 
preparation of 
materials) 



100 
200 



80 
40 



320 
160 



Individual conferences^ 
(office hours) 



20 
40 



20 



80 



100 



50 
100 



190 



200 



Small-group work^ 
(30 per group) 



100 



Total 
man-hours 



420 



480 



900 



380 



540 



900 



* lAboratory— 4 hours per week, 5 laboratory groups in a conventional programme. Lat>ora* 
tory available at scheduled hours for studenu' use, staffed by assistants only In non* 
traditional organltation. 

^ Time necessary to Identify or prepare texts* articles, or self*testing devices for 
students to use Independently at home, In the library, in the laboratory, or in the 
classroom. 

« Provision for each student to have two 30>minute Individual conferences during the 
20>week course. 

* Provision for one-hour smail-group conference every other week for each group of 
30 students. 
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facilitated.^ But structure can impede that process 'id the accom- 
panying table offers an illustration of how a specific lecture/laboratory 
programme for 300 students might be modified to achieve the goal of 
greater individualization without any change in the amount of faculty 
time devoted to the educational tasks. There is ample evidence to 
support the view that such practices are effective.^ The question of 
whether teachers are willing to adopt them cannot be answered with 
a comparable degree of confidence. Unless they do, however, it is 
unlikely that the educational problems of teaching large groups will 
ever be solved. 



1 Rodin. A.E. A Levinc. H.G. (1973) /. med, Educ,, 4S» 349. 

* PostlethwAite, S.N. (1965) Ptant sclencet a workbook with an audio program approach, 
Minneapolis, Burgess* 
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THE EVALUATION OF TEACHERS 
AND TEACHING EFFECTIVENESS 



Christine H. McGuire^ 



Fundamentally, the goals of education are not unlike the goals of 
therapy, i.e., to facilitate optimum functioning of the organism. Hence 
the methodology for evaluating teachers and teaching is, in essence, 
analogous to that employed in evaluating any other therapeutic agent, 
and the pitfalls of research in this area parallel those in any field 
of clinical research. 



A DESCRIPTION OF CURRENT APPROACHES 

The three approaches to the evaluation of teachers and teaching that 
are currently most widely employed and that appear to hold the greatest 
promise can be distinguished in terms of the primary object towards 
which assessment efforts are directed: perceptions, pr '^ess, and product. 
As sliown in Table 1, each approach has its analogue in clinical research 
and, like the latter, its characteristic data type and source. The methods 
of collecting these data, their interpretation, and their limitations are 
outlined below. In this brief synopsis, discussion is limited to a definition 
of the approach and the identification of the question or questions each 
is designed to answer. 

Assessment of learner perceptions 

Although the oldest, this approach is stiU widely used and very 
commonly misused. It is designed to answer 'be question: "How did 
students feel about the quality of the teacher, the teaching, and the 
programme?" Formal or informal student reactions and/or ratings 



^ Associate Director, Center for Educational Development, University of niiools CoQete of 
Medicine, Chicago, IlL. USA. 
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TABLE 1. APPROACHES TO EVALUATION OF 
TEACHERS AND TEACHING 



Approach 



Data source 



Clinical equivalent 



Assessment of 

perceptions 



Reports of student 
reactions 



Evaluation of patient* s 
subjective feelings 



Assessment of 
process 



Observations of 
student-teacher 
Interaction 



Evaluation of therapeutic 



rationale on patho- 



physiological principles 



Assessment of product, Tests of student Evaluation of therapeutic 



constitute the data for answering the question. Typically, these data are 
collected by interview and/or questionnaire that may range from highly 
structured to completely open-ended inquiries. These may be couched 
in a form that invites the student to comment directly on his reactions, 
or in a form that induces him to respond to a series of indirect probes 
in a manner that reveals reactions about which he would be less able 
or less willing to be candid. 

Assessment of process 

In contrast to the assessment of perceptions, which deliberately seeks 
opinions about the merits of various elements in the instructional 
programme, process assessment is designed to determine what actually 
occurred in the encounter between teacher and learner. The basic 
question that it attempts to answer is: "What was the nature and quality 
of the setting and of the communication (interaction) between student 
and teacher?" 

The information needed to answer that question is simple descriptive 
data obtained from systematic, impartial, skilled observation of the 
teacher and the learner in the instructional situation. Since, however, 
there is an unmanageably large number of attributes of the learning 
situation (ranging from the temperature of the room to the "heat" of the 
student— teacher exchange) on which observations could be made, it is 
necessary to select those characteristics likely to be most important in 
facilitating or inhibiting achievement of the desired outcomes. This 
decision is, at least implicitly* based on a theory of Icaming, and thus ' 
the observations made are, in fact, designed to answer the question: 
'To what extent does the teacher create an environment in which the 
actual conditions for learning are congruent with those that a selected 
theory of learning asserts as optimal?'* 



outcome* or effect 



performance 



efficacy 
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Assessment of product 



Whether a recommended set of conditions for learning is, in fact, 
optimal can be determined only by examining their consequences. It is 
to this issue that the assessment of educational product or outcome is 
addressed. All product assessment is designed to answer the question: 
*To what extent is the educational agent effective in accomplishing the 
educational goals?" If it is true that teachmg is undertaken mainly 
for the piupose of producing student learning, it follows that teacher 
effectiveness must ultimately be assayed in terms of the extent to which 
this intended outcome is achieved, and teacher efficiency judged only 
after the costs are docu^'iented and charged against the changes in 
student knowledge, judgement, skills, habits, and attitudes that are 
realized 

Two types of data are therefore indispensable in answering the 
question posed in this approach: those derived from comprehensive 
measures of relevant aspects of student performance, and those obtained 
from careful cost accounting of teacher and student time expended, 
other resources consumed, and undesired side effects induced in the 
educational programme. 

THE MEIHODOLOOY OF CURRENT APPROACHES 

Assessment of perceptions 

The basic data on student satisfaction are variously obtained: some 
institutions employ a standard rating form in which students evaluate an 
instructor or a course, ranking each in relation to others he has known; 
other institutions use a more detailed questionncdre designed to elicit 
candid reactions to multiple elements (e.g., a particular lecture, the 
required text, tutorial hours, etc.) cf the instructional situation. With 
respect to each, students may be asked to indicate what they Uked or 
disliked and the intensity of that feeling, what was found valuable or 
useless, what experiences they would prefer to have expanded or 
reduced, and the like. Both rating forms and questionnaires are 
often combined with a third type of student reaction form designed to 
probe specific feelings that actions of the instructor have evoked In 
general, such questions (illustrated in Fig. 1) are sdected on the baas of 
contemporary learning theory and represent those aspects of the 
student --teacher relationship thought to influence the efficiency and 
effectiveness of learning. 

Data collected from the ratings, questionnaires, and other reaction 
forms are sununarized to show general tendencies in student evaluations. 
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FIG. 1, EXCERPTS FROM A STUDENT REACTION FORM 



Did you feel free to challenge the instructor when you thought some- 
thing he said was incorrect? Yes No 

Did you feel free to ask the instructor a question about something you 
did not understand? Yes ... No 

How often were you stimulated to think about additional applications of 
concepts and principles the instructor was discussing? Practically 
always . . Occasionally . . . Rarely .. . . Never 

Did you find that the instructor was sufficiently flexible in modifying 
his plans to suit your needs? Yes No 



When possible, responses are correlated with intellectual and personality 
characteristics so as to enhance their usefulness in faculty efforts to 
individualize instruction. 

Process assessment 

Broadly speaking, two quite different rationales and associated 
methodologies characterize most forms of process assessment: (1) 
analysis of the conditions for learning samples the elements that 
characterize the total learning situation; (2) interaction analysis focuses 
only on those elements (primarily verbal) that characterize teacher 
behaviour and/ or student— teacher interaction. 

An analysis of the conditions of learning begins by identifying as 
concretely as possible the outcomes sought in a particular instructional 
situation — a course, a lecture, a seminar, a laboratory, ward rounds, 
and the like. For example, the general outcome of ""developing skill in 
physical examination of the patient" could, if specified by a detailed 



I Learning tends to be more efficient when the objectives are clearly 
I understood by the learner. 

Learning of a skill is facilitated by frequent, brief practice periods spaced 
over an extended time period. 

Prompt, specific feedback that enables the learner to identify errors in 
[ his performance facilitates the teaming of a skiiL 

Prompt specific feedback that reinforces correct elements of a 
performance facilitates learning. 



FIG. 2. SELECTED PRINCIPLES OF LEARNING 
RELEVANT TO THE LEARNING OF A SKILL 
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definition of tbe components of that skill, be used as the starting 
point Once the desired outcome has been determined, the next step 
must be to identify the learning principles that apply most direcdy to 
the specified learning goal. For the illustration given above, the 
principles stated in Fig. 2 would be among those isolated for considera* 
tion. Using such principles as a frame of reference, an evaluator then 
proceeds to construct a higihiy structured questionnaire, interview, or 
observation form that will yield objective, descriptive data about the 
extent to which the specified conditions are met in the actual situation. 
An excerpt from such a form, developed for the illustration cited above, 
is shown in Fig. 3. Note that, with minor changes, the questions listed 
there could be employed as a checklist by an observer, as an interview 
schedule by a trained interviewer, or as a structured questionnaire for 
direct distribution to students. In whatever mode they are emjdoyed, 
the two essential elements are: (1) that each question be derived from a 

FIG. 3. EXCERPT FROM A FORM DESIGNED TO ASSESS 
THE CONDITIONS FOR LEARNING A SPECIFIC SKILL 



Were the objectives of the course (the instructional unit, the instruc- 
tional exercise) explained to the students? 

What is the evidence that these objectives were actually understood by 
the students? 

How much opportunity did the students have to practise Mch 
manoeuvre involved in the physical examination of the patient? How 
frequentiy were these practice periods scheduled? When (in relation 
to the relevant didactic material) were they scheduled? How long 
was each practice period? Were variable opportunities provided to 
students in accordance with individual differences in proficiency? 

How often did the instructor actually observe the student examining 
a patient? 

Did the instructor provide any -eedback to the student on the accuracy 
of his findings? If yes* when was the feedback provided? In what 
form was it provided? 

Did the instructor provide any feedback to the student regarding his 
technical skill in eliciting these findings? If yes, when was this feed- 
back provided? In what form was it provided? 



principle of learning that is relevant to the educational outcomes sought; 
and (2) that each question be formulated in a manner that will yield 
objective, descriptive information, not opinions or preferences. 

Finally, it should be noted that, since communication of some kind 
is inherent in any instructional situation, assessment of the conditions for 
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learning will also include at least a few generally applicable questions 
that yield data about the basic characteristics of the communication. 
This evidence would be derived from such questions as, for example: 
**Could the speaker be heard by all parties to the communication?"; 
**I>id he use terms that were understandable to his listeners?"; "When 
introducing a new term did he define it?"; "Were abstract concepts 
illustrated with concrete examples?"; "Conversely, were specific bits of 
illustrated with concrete examples?"; "Conversely, were spedfic items 
of information and data related to a general frame of reference?"; 
"What variety of media was used to supplement the spoken word?"; 
Responses to questions such as these yield the necessary data for 
determining whether the most elementary requirements of the com- 
munication process have been met, and thus whether there is some 
probability that communication has occurred. 

In contrast with the approach described above, the various forms of 
interaction analysis focus exclusively on the nature of the communica- 
tion process. This is accomplished durough detailed, systematic analysis 
of the behaviour (predominantly verbal) of teacher and/or sbdents. 
Originally developed for use in a formal classroom situation, most forms 
of interaction analysis are also useful in other instructional settings. 
Some adherents of this technique (e.g., D. G. Ryans, B. O. Smith*) focus 
attention primarily on teacher behaviour; others (e.g. R.F.. Bales, 
N. A, Flanders and E. J. Amidon^ use the method to examine the 
quantity and quality of interaction between teachers and students. 

Irrespective of the precise emphasis, the general methodology is 
similar for most variants of this approach. Specifically, observers are 
trained to record at specified interv^ (e.g., every 3 seconds), on a form 
(such as that shown in Rg. 4), the nature of the verbal behaviour then 
occurring. These recordings are made for predefiaied periods (e.g., 
10—15 minutes) and the periods are selected to sample different points 
(e.g., the beginning, middle, and end) during an instructional session or 
during several sessions extending over a school term. The recordings are 
then summarized and analysed in terms of such variables as percentage 
of teacher talk as compared with students talk, percentage of teacher 
talk in each behavioural category, percentage of student talk in each 
category, characteristic sequences of teacher— student behaviour, and 
the like. 

Certain proponents of this approach define the variables to be 
observed in strictly behavioural terms (as in Fig. 5) in a manner that 
requires the observer to make only minimal interpretation. Others, in 



^ Pi Lambda Theta (1967) The evaluation of teaching, Washington, D,Co Banta. 
« Am^don, E. J. A Houah. J. B, (1967) Interaction analysis: theory, research and appUcatton, 
Reading, Mass., Addlsoo-Wesley. 
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FIG. 4. EXCERPT FROM A FORM FOR RECORDING 
STUDENT -TEACHER INTERACTION 



INSTRUCTIONS 


Uslna thA svp^ibols shown below, make a notation 




Ax/firv throo (S\ ftAronri^ for ten (10) consecutive 




mtniitA^ to InritrfltA the nature of the behaviour 




occurrlna at oach thrAA>^AConH interval. 


LEGEND 


Teacher behaviour: 




1 = asKS a quesiion 




2 = aivds Instructions 




3 zz Droi^des information 




A zz reflects student comment 

9 vll W tw Wli^ WWI III II VIII 




5 = etc. 




Student behaviour: 




zz askfi a nu Afition 




7 = answers a question 




8 = provides information 




^ — oil*. 








10 = silence 




11 = etc. 


Date: 


Hour: . . Class: . 


Instructor: 


Observer: 


First recording-time : 



an attempt to get at more dynamic elements in the interaction and 
to identify potentially facilitative and inhibitory roles played by each 
participant, include such terms as "clarifies", "summarizes", 
"obstructs", "expresses hostility", and "challenges" in the list of 
behaviours that the observer is to record. Inclusion of such terms 
requires more interpretation of the observed phenomena, introduces 
greater subjectivity into the observations, and imposes the need for 
more extended training of observers, in order to achieve acceptable 
levels of inter-observer reliability. However, irrespective of the specific 
categories of teacher and/or student behaviour selected for recording, 
this approach is explicidy lunited to observation of the interaction 
process per se; no attempt is made to relate that process to objectives 
or to outcomes, either direcdy or indirecdy through general learmng 
theory. 
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Product assessment 



In contrast, product assessment in its pure form is concerned 
exclusively with objectives (product sought) and outcome (product 
realized), and thus requires that the goals of the educational pro- 
gramme be clearly specified, that they be defined in terms of 
behavioural changes to be brought about in the learner, that test 
situations be designed to sample these behaviours, and that these test 
situations be administered to the learner to determine whether he is 
able to perform in the desired manner and at the prescribed level of 
competence. 

The techniques used to assess performance range from conventional 
tests of information, through sophisticated simulations of clinical 
problems, to long-term, systematic observations of the learner in varied 
professional settings. These techniques, extensively described elsewhere,^ 
do not differ in kind from those regularly employed to certify student 
competence. Only their scheduling and the method of suro^iarizing 
their results need be modified when performance tests are administered 
to students for purposes of teacher, rather than student, evaluation. 
Since in the former case the emphasis is on the change brought about 
in the behaviour of the learner, it is necessary to administer the same 
or comparable tests at the hennaing and again at the end of the 
instructional period under scrutiuy. Further, if long-term effects are 
included in the objectives of the programme, a complete evaluation 
requires follow-up data regarding subsequent performance of the 
learners. 

Just as the timing of any assessment is determined by the purpose for 
which it is made, so must the method of summarizing the data be 
dictated by the purpose for which they are collected. In certifying 
student competence it is adequate to report merely whether the student 
does or does not meet the required performance standards. For 
programme evaluation it is also important to know how many students 
do or do not meet each of these standards. Indeed, if large groups 
persistently fail to meet any of them this is a clear sign that something 
is wrong somewhere — in the teaching, the students, the test, or the 
standards. However, for purposes of evaluating teaching such a sum- 
mary is insufiBcient. 

To serve this purpose the performance data must be presented in a 
manner that helps to answer the following questions: (J) How much 
gain was made by the total and by relevant subgroups of students?; 
(2) What are the characteristics of the students who appear to gain 



^ Sec Charvat, J. ct al. (1968) A review of the nature and uses of examinations in medical 
tucatlon, Geneva, World Health Organization (Public Health Papers, No. 36). 
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most (or least) from the current instructional programme?; (3) What 
are the specific areas (e.g., concepts, skills, professional demeanour) of 
greatest and of least gain?; (4) What specific misconceptions (or other 
forms of madequacy) are still relatively common among students at the 
end of the instructional programme? 

Qearly, conventional student performance reports showmg the 
distribution of scores at different pomts in time yield very little evidence 
that is useful in answering such questions. Only detailed item analyses 
of the results of each administration of the performance tests will 
suflBce. Such analyses can be designed to report the number ot students 
in each group and each relevant subgroup who ^ve each possible 
response and who make each type of error, and ixr changes over time 
in this performance pattern. Such information is particularly valuable 
when supplemented by various item statistics that report the correlations 
between each response and appropriate criterion variables, and between 
each response and relevant student characteristics anu/or programme or 
instructional variables. Techniques of summarizmg performan^je data in 
this manner are explained in standard texts on test analysis. 

No product evaluation is complete without cost analysis, particularly 
since evidence is mounting that, ^ven adequate time, personal instruc- 
tion, and self-instructional resources, virtually all students of normal 
intelligence and emotional stability can master almost any element 
among the objectives of higher education. The issue therefore becomes 
an economic one of determining whom a country can afford to educate 
and at what level of mastery, i.e., at what cost in resources. Given the 
enormous attention currently devoted to the economics of health care 
delivery, it is astonishing how little consideration is being given to the 
economic issues of health professional education. Most presently 
available data are deficient on two counts: (1) they are chiefly 
retrospective; and (2) the data categories are defined in such gross terms 
that it is usually impossible to allocate costs along genumely functional 
lines. 

In order to evaluate teaching and teachers in cost -benefit terms, a 
detailed recording of the amounts of all resoxurces that go into any 
instructional effort is urgently needed. K we are to avoid the notorious 
unreliability of retrospective data, this implies investigating represen- 
tative samples of students and faculty to determine how much of their 
time is actually devoted to some kmd of instructional activity, the exact 
nature of that activity, and the way in which the quantity and distribu- 
tion of such activities change with different instructional methodolo^es. 
It also suggests the importance of costing out other elements in the 
instructional situation, such as, for example, the additional laboratory 
and hospital costs where patients are used as "teaching material". At 



present the only usable data tend to be those concerned with the direct 
money costs attributable to the construction and maintenance of special- 
ized facilities (c.g., lecture halls, student laboratories) and materials 
(educational films, slides, self-instructional programmes). 

In the absence of hard facts, it is difficult to escape the suspicion 
that, because their costs are hidden, we continue to cUng to traditional 
methods that are not only meffectual but may also be inexcusably 
uneconomic, for they rely on the most costly resources of all pro- 
fessional time and patient disability. 



USES AND LIMITATIONS OF CURRENT APPROACHES 

Since, if properly interpreted, some data are usually better than no 
data, it could be argued that the primary Umitation of all current 
approaches lies in their madequate usage. However, each one also has 
certain additional shortcomings that must be assessed if serious errors 
in interpretation are to be minimized. 

Assessment of perceptions — the Idiocy that truth is in the eyes of the 
beholder 

Qearly, when perceptions are used as the primary data for evaluating 
teachers or teachmg, there is always the danger that student reactions 
will be interpreted as if they were statements of facts, rather than highly 
subjective, individual, and variable opinions. Thus, when a student says: 
"The programme was very valuable; I learned a great deal," there is a 
natural tendency to believe this to be so. Unfortunately, there is no 
evidence to support the view that enjoyment, satisfaction, hard work, or 
any other reported "positive" reaction is necessarily correlated with 
learning. On the other hand, there is also no evidence to support the 
view that something distasteful (either medical or educational) is neces- 
sarily "good" for the recipient Nonetheless, provided the data are 
properly interpreted as simple statements of consumer satisfactions and 
dissatisfactions, the assessment of teaching through student perceptions 
can (1) assist in the diagnosis of some causes of inadequate achievement; 
and (2) contribute to flie identification and creation of conditions that 
will enhance motivational and other affective responses that facilitate 
learning. 

Assessment of process — the fallacy that fads and fetishes produce 
results 

The history of therapeutics is replete with examples of interventions 
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earnestly advocated aod consistently employed by one generation of 
physicians, only to be proved useless or even harmful by a subsequent 
generation. Widespread adoption of the current "treatment of choice", 
in the absence of suppwtive data about its relative eflBcacy, epitomizes 
the pitfalls of a preoccupation with process assessment, independent of 
product assessment For, like therapeutics, education is a victim of 
process fads. 

While educational science has been successful in demonstrating that 
teachers are intolerably costly and inexcusably inefQcient purveyors of 
information, or dictators of concepts more accurately recorded in readily 
avaOable literature, it has unfortunately not been so successful in 
defining the teacher^s role positively. Repeated experimental studies of 
the relative effectiveness of varied instructional methodologies — 
lecture, discussion, programmed instruction — have produced conflict- 
ing evidence and inconclusive results. It may have been naive to expect 
any definitive outcome from experiments that, on reflection, appear to 
represent a sunplistic search for an educational panacea — a pursuit no 
less futile than Che quest for an all-purpose, miracle drug. 

In rejecting the role of lecturer and accepting the view that mature 
students are responsible for their own learning, however, some teachers 
have gone through what can only be described as an "identity crisis". 
In this anxious state many have abdicated all responsibility for defining 
objectives, planning the educational programmes, creating appropriately 
stimulating conditions and adequately varied resources, setting standards 
of student performance, and gathering the requisite data for ensuring 
that students meet those standards. Regrettably, there has been little 
evidence that adherents of the newer and more sophisticated forms of 
process assessment have attempted to solve this identity crisis by 
utilizing their potentially powerful tools in systematic studies of the 
relation between teacher behaviour and student achievement 

Until the basic connexions between process and product are estab- 
lished, the utility of process assessment must remain limited to research, 
training, and diagnostic applications. As a research tool, process 
analysis opens up the opportunity of determining the relative 
effectiveness of alternative elements in teaching strate^es; as a trainmg 
tool it enables the teadier to hold a mirror to his performance and to 
determine the extent to which his actual conduct was a true reflection 
of his intended strategy; as a diagnostic tool, process assessment, when 
used in conjunction with product assessment, yields data that may be of 
value in suggesting alternative hypotheses to explain actual outcomes 
that fall short of intended outcomes. At this stage of educational 
development, process assessment cannot be used in direct evaluation 
of teachers and teaching effectiveness; it can best be used only in 
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conjunction with other methods to assist teachers m improving their 
performance. 

Product assessment — the fallacy of confusing concurrence with 
causality 

Although product assessment furnishes the only data that are directly 
relevant in the ultimate evaluation of teaching effectiveness, it also 
presents the greatest diflSculties m fundamental study design, for four 
reasons: (1) all the factors that normally confound any research on 
human subjects are present in all studies of educational outcome; (2) 
the study population is never randomly selected and its small size and 
lack of comparability with other populations undergoing alternative 
educational treatments make generalization hazardous, even when 
conventional statistical safeguards have been applied; (3) the teachmg 
being evaluated usually constitutes such a small proportion of the total 
educational influences operating on the student subjects that it becomes 
almost impossible to establish plausible causal linkages; and (4) the usual 
controlled experiment (even less, the double blmd) design is rarely 
feasible and almost never suitable. Indeed, Glaser asserts that the 
tyranny of the controlled experiment has probably done more to retard 
educational progress than any other single influence. He bases this 
conclusion on the observation that the only issues that can be studied by 
this method are so trivial that they are hardly worth the effort, that the 
findings are available only at a pomt m time when they are useless for 
decision-making, and that the requirements of the design are such that 
the conditions to which the findings apply can rarely be duplicated. 

In addition to these inherent limitations, product assessment imposes 
one additional hardship on the conscientious evaluator: the necessity of 
estimating all elements of the product that are relevant to a particular 
instructional programme. Addre.ssmg a very limited range of objectives 
(e.g., increase in factual knowledge) continues to be one of the most 
common failings of studies utilizing this approach. 

Given such severe limitations, it may appear that product assessment 
is of little practical utility. However, at the same time, it is the one 
indispensable approach to the evaluation of teaching. This apparent 
paradox is readily explained: although it may be impossible to establish 
by this method that a given strategy is responsible for success, it is the 
only method by which failure to achieve the goals of teachmg can be 
documented and the warning clearly sounded that "somethmg is 
wrong". Other approaches can then be employed in the effort to identify 
and remedy the deficiency. 
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CONCLUSION 



Dissonance between the results yielded in the assessment of product 
and those obtained from the assessment of process and perceptions is 
sufficiently common to impose the necessity of collecting a variety of 
types of data as a basis for making inescapable decisions about the 
effectiveness of teaching. Three approaches to the evaluation of 
teaching have therefore been described as sources of the required data 
The methodology of each has been presented, together with a 
discussion of its utility and its characteristic limitations. In the light 
of these limitations a combination of all three approaches is recom- 
mended as indispensable. 
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medical education, Geneva, World Health Organization (Public Health Papers, 
No. 36) 

Chapter 4 provides a succinct summary of new techniques for product 
assessment and a description of certain approaches to process assessment other 
than interaction analysis. 
Gage, N. L., ed. (1963) Handbook of research on teaching: a project of the 
American Educational Research Association, Chicago, Rand McNally 
Only for the reader who desires a comprehensive, sophisticated discussion of 
the merits and limitations of alternative research designs. That reader will 
find this volume an extraordinarily valuable resource in suggesting alternative 
solutions to the problems of research design in this area. 
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MICROTEACHING FOR TEACHER TRAINING 
Richard P. Foleyi 



In response to student dissatisfaction with traditional modes of 
trainings medical educators everywhere have been engaged in thoughtful 
examination of faculty training programmes, since it has become 
increasingly evident that more systematic and economical methods are 
needed for evident teachers to acquire increased pedagogic skill. 

One alternative to present teacher-training methods that has been 
widely applied by educators from other disciplines is microieaching. It 
is the purpose of this paper to describe the technique conceptually and 
operationally, and to offer suggestions for the incorporation of the 
metliod into the programmes of schools for the health professions. 

MICROTEACHING: A DESCRIPTION 

A graduate student in microbiology holds up an agar plate before 
four first-year medical students to demonstrate its use in isolating 
bacterial colonies. Fifteen feet away, a professor of microbiology who 
is supervising the teacher aims the camera of a videotape recorder at 
the group and occasionally writes down some notes. Although the 
teaching lasts for only 10 minutes, two significant thmgs happen. The 
medical students have had an opportunity to learn one method for 
isolating bacterial colonies, and the teacher has had an opportunity to 
practise the instructional technique of presenting a laboratory demon- 
stration. yVfter the presentation is concluded, the supervisor asks each 
student to fill in a questionnaire regarding the teacher's performance. 
The students then leave the room and a 15-minute critique follows in 
which the supervisor and instructor discuss the demonstration, review 
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the written comments of both supervisor and students, and view 
segments of the videotape. After a short break, during which the 
practice teacher has the opportunity to incorporate suggestions from the 
critique, the entire process is repeated with a different group of medical 
students. The teacher, the supervisor, and the students have all been 
involved in a microteaching exercise. 

Microteaching is designed for teachers who want to mprove their 
teaching skills. The key elements of this training method, illustrated in 
the above example, include: 

(1) A controlled, constructed setting for producing instructional 
skills. By reducing the length of the presentation, the scope of the 
content, and the number of students, the graduate student in micro- 
biology can practise an instructional skill, such as ^ving a laboratory 
demonstration, in a setting free of many of the complexities of the 
actual classroom. 

(2) Feedback from multiple sources. After pving the 10-minutc 
microlesson the teaching assistant is provided with suggestions or 
feedback for improvmg his performance from a content expert acting 
as the supervisor, from the students he has taught, and from a videotape 
playback. 

(3) The opportunity for immediate integration and application of 
feedback. After the feedback has been received there is an opportunity 
to integrate and apply these suggestions in reteaching to a different 
group of medical students. 

Five essential propositions underlie microteaching as a training 
method: 

1. The most effective way to master the skills of teaching is to practise 
one skill at a time 

Too often a teacher may be informed by his department chairman, 
students, or colleagues that he needs to "liven up his lectures** or "allow 
students more opportunity to participate**. Such advice is rarely hdpful, 
for if the teacher knew how to deliver an exciting lecture or increase 
student involvement he would probably be doing so. 

In microteaching, an individual willing to improve his instructional 
effectiveness concentrates on the objective of practising a single instruc- 
tional skill, mastery of a unique set of curricular materials, or 
demonstration of one teaching method. Focusing on specific objectives, 
rather than the teaching*Ieaming process in general, allows a teacher to 
discriminate among the elements of teaching, and assists him in 
practising a particular technique and incorporating it in his behaviour. 
For example, if the objective of a health educator were to increase 
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student participation in the classroom, be mi^t concentrate on using 
positive verbal reinforcement (such encouraging comments as ""Good**, 
^'ExceUent^ or other statements indicating satis&ction with the 
re^XMise); or non-verbal reinforcement (such actions as nodding his 
head aflSrmatively, smiling^ or moving towards the student). 

By concentrating on a particular skill, a teacher can begm to recognize 
the behavioural components of ^t technique and the situational factors 
that dictate thdr use. For example, the health personnel educator who 
has learned what kinds of verbal and non-verbal cues constitute 
reinforcement and how to use them can then employ this skill to bring 
about m ax im u m participation* When a teacher has mastered a variety 
of instructional techniques and knows the effect of each, he can apply 
them to achieve instructicmal aims in a planned rather than a chance 
arrangement 



2. The complexities of the actual classroom situation limit its value in 
the training of teachers 

Most teachers learn how to teach on the job, but classroom practice 
has significant limitations. First, there is simply too little opportunity 
for trial and error. A beginning teacher, in particular, is often anxious 
about making a good impression on his students. Consequently, if his 
first lecture is a success and his first group discussion a failure, the 
chances are that he will become committed to the lecture method of 
instruction as the most effective way to teach. Secondly, there is only 
limited opportunity for a teacher to receive feedback on his performance 
in the dassroom, and even that is rarely sought Finally, and most 
important, students are in a classroom to be taught effectively, not to 
be practised on. 

Microteaching is designed to provide a safe setting for the acquisition 
or refinement of educational skills and techniques. A be^nning teacher 
mig^t find delivering a lengthy lecture to a large body of students an 
awesome task. Practising this method of instruction in a microteaching 
setting, where the class size, the time, and the scope of content are 
reduced, considerably alleviates this anxiety. Rather than deliver a 
SO-minute presentation to a group of 75 students, he can concentrate on 
one aspect of the lecture, perhaps the introduction only, and practise 
this component in a S-lO-minute microlesson to a group of 4-5 students. 
Having mastered the introduction, he is ready to practise the body of 
the lecture and the conclusion. Since the students observing the lesson 
would usually not b the teacher's, he is freer to experiment and less 
anxious about receiv. j student feedback. 
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3. The more knowledge a teacher has of his performance during a 
practice session, the more valuable the experience 

Supervision is generally equated with evaluation and is therefore 
regarded by many teachers as something to be avoided. This is 
unfortunate, since constructive supervision can be one of the best 
resources in a teacher's professional development 

Microteaching greatly increases the opportunity for feedback. Since 
the emphasis is on practising a specific skill or achieving some other 
predse objective, the teacher, the supervisor, and the students have a 
clear picture of the purpose of the microlesson before it is presented- 
As a result, the critique that follows is concerned not with the overall 
performance, but with only one aspect of teachmg behaviour. This 
approach is generally less threatening and results in more specific 
recommendations for improving teaching. 

The several sources of feedback are designed to provide a teacher 
with maximum insight into his performance in the light of his goals. 
The teacher and the supervisor together review student comments, as • 
well as video or audio playbacks, and analyse the way in which the 
performance might be further improved. This feedback can then be 
inmiediately translated into practice by reteaching the lesson to a 
different group of students. The emphasis in microteaching is on helping 
a teacher to improve, not on jud^g his performance. 

4. A training method that can be varied is superior to one that is 
rigid 

The most prevalent model for training bediming teachers takes the 
form of education courses supplemented by practice teaching in which 
they observe and work with highly competent "master teachers", with 
the expectation that practice teachmg ^ help the student's transition 
from die theory to die "real world'* of teaching. 

This model has grave limitations. Its value for the tramee depends 
both upon the expertise of the master teacher and his relationship with 
the trainee. The supervisor simply may not allow much opportunity 
for die beginning teacher to practise, or may not provide him with 
adequate supervision. Even in the ideal relationship, however, there are 
drawbacks. For example, the master teacher cannot interrupt the class- 
room session to offer constructive suggestions, and there is no 
opportunity for the trainee to stop his presentation and repeat some 
portion that needs improvement Also, practice in the classroom must 
be integrated into die flow of die subject-matter: die skill or technique 
being practised must fit well with the content specified for that day. 

Microteaching, on the other hand, allows for maximum control of the 
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practice session* Hie length of the teaching, the number and kmds of 
students, and the method of feedback can be varied accortfing to the 
goals of the particular session. For example, the teacher, referred to 
earlier, who is interested in improving his skills in lecturing, can, after 
gaining comfort in presenting a lO-minute lecture to 4— 5 students, give a 
20-minute lecture to 10—12 studenrs, thus more closely app'-oximating 
his teaching in the actual classroom* Unlike conventional prcctice 
teaching, microteaching provides and opportunity to practise the skill of 
lecturing until it is mastered, using videotape and written comments 
from the supervisor as a record of progress. 

Since a number of variables can be easily manipulated in this 
constructed setting, microteaching is also an excellent research method 
By mampulating a sin^e variable, such as the nature of the supervision, 
countless hypotheses related to the improvement of teacher effectiveness 
can be investigated Mdeotapes and the written feedback supply the 
researcher with recorded data from which he can work. 

J. A genuine teaching experience is superior to an artificial one 

Peer teachmg, role playing, and other simulated teaching experiences 
are being widely applied in teacher-training programmes. Alth ugji 
these methods do offer better opportunities for teachers to practise skills 
than the classroom, there has b^n a good deal of questioning about the 
transfer of learning from these experiences to the actual teaching 
situation. Fundamental to the microteaching concept is the fee* inat it is 
real. Although the setting is constructed, bona fide tearlung of real 
students occurs. 



INCORPORATING MICROTEACHING 
INTO THE PROGRAMMli OF A SCHOOL FOR THE HEALTH PROFESSIONS 

Health personnel educators face the dual task of improving the 
msinictional skills of existing faculties in schools for the health 
professions and of developing these skills in graduate students about to 
begin teaching. Peihaps one of the soundest and most economical routes 
for achieving both ends is to train the two groups simultaneously, using 
microteaching as the vehicle. 

Where to begin 

Graduate students in both basic and clinical sciences frequently 
function as teachers under an experienced faculty member as part of 
their academic work, but rarely does any formal training in education 
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accompany this txpeneoce. This situation provides sn excellent 
oppoiPsmty to introduce microteaching as a method for assisting these 
students to acquire the necessary instructional skills. Short-term 
programmes could be designed to increase their compete&'^e in 
conducting lectures, demonstrations, or ffoup discussions. Since these 
programmes would not necessarily have to constitute a formal, academic 
course, both established faculty and graduate students mi^t be offered 
this opportunity. Hie feculty members could be asked to present a live 
or videotaped lecture or laboratory d^onstration to serve as a model 
for introducing the graduate student to these techniques, to function as 
supervisors in microteaching sessions, or to serve as consultants in 
deigning the training programmes. 

Designing and implementing a microteaching training programme 

One of the most important decisions in dedgning a microteaclung 
programme is identification of the specific techniques or strategies to be 
practised by the participants. Snce training in instructional skills that 
are not relevant to trainee needs is generally wasted cRorty it is {mident 
to involve potential participants in the planning stage througjh interviews 
oc questionnaires that can quickly isolate those instructional areas in 
which they feel some need for assistance. 

An expert in education is needed to as^t participants in this 
identification, to provide the educational content of the programme, and 
to oversee the operational details of the microteaching setting. For a 
small programme with IS-* 20 trainees, one educator and 3—4 faculty 
supervisors familiar with the content of the programme and trained in 
the microteaching process are sufiBcient If videotape recorders are 
used, an operator must be available to man the equipment If the 
supervisor can be spared this task, it permits him more time to concen- 
trate on the teacher's performance. If not, supervisors can be quickly 
trained to operate the equipment For a larger training programme, 
staff should be increased accordingly. 

Although several options are available for introducing this training 
activity to participating teachers, it will be most productive if at least 
one day before the microteachio^ programme begins participants read 
the material that describes the sldU to be practised; take part in a 
discussion that leads to the formulation of those criteria that 
demonstrate effective use of the skill; view a live or videot2Q)ed 
presentation of the sldU to which they can apply the criteria; design 
the evaluation forms to be used by students and superviscnrs; plan a 
microlesson that incorporates the skill to be practised; and become 
familiar with such operational details as the expected length of thdr 
lesson, the time of the reteach, and the kinds of students to be taught 
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Since oiicroteacking is real teaching, students must be recruited who 
approximate, as nearly as possible, to those that the teacher ordinarily 
encounters. If, for example, one is conducting a microteaching 
programme to improve the questioning skills of teachers respon^ble 
for a basic science programme in a medical school, then htpnimg 
medical students would be the most suitable choice. 

Although the number in a microteaching session can range from 
1 to 10 students, depending upon the purpose of the lesson, 4 or 5 
students is usual In deciding upon the number it is well to keep in 
mind that microteaching is designed to assist teachers, not students. 
The setting, therefore, should be one in which teachers feel free to 
experiment without fear of faSure. Unless a large number of students 
can be justified, the groups should be kept small so that concentration 
of the skill and maximum feedback can be achieved. 

There is no prescribed length for the microlessons; they could vary 
in length from 3 to 30 minutes, depending upon the goals of the 
programme, the number of participants, and the time available. The 
most frequent length is 5—10 minutes, since this period generally afibrds 
adequate opportunity to practise a particular skill and to cover a 
significant amount of content 

Similarly, the length of the critique may vary. Ten minutes is 
generally adequate to allow feedback from students, supervisors, and 
videotape. With the format of a S-minute teaching session and 10-minute 
critique, two teachers can complete a teach-critique-reteach-critique 
cycle in an hour. For example: 



9.00-9,05 


teacher "A"— teach 


9,05-9.15 


critique 


9.15-9,20 


teacher "B"— tcadi 


92(^930 


critique 


9J0-9J5 


teacher "A"— reteach 


935-9.45 


critique 


9.45-9J0 


teacher "B" — reteach 


9JO-10,00 


critique 



Although the length of time between the teach and the reteach can 
be varied, if the reteatii is given immediately after the first critique 
there is generally not sufficient time for the teacher to incorporate in 
reteaching the suggestions offered. On the other hand, reteachhig a day 
or two later we^ikens the effect of the feedback received front the 
critique. An mterval of IS minutes to an hour between the teach and the 
reteach enables a teacher to plan a second lesson with the feedback 
suggestions freshly in mind 

Supervision is of critical importance to successful microteaching and 
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merits the following considerations. First, as the focus of the critique 
increases, its effectiveness decreases* If more than two sugg^tions are 
offered for chanpng performance, a teacher often becomes confused^ 
defensive, or discourage Secondly, supervision should be supportive 
and lead the trmnee towards self-diagnosis. This can best be achieved 
by allowing time for the subject to comment upon his own performance. 
The supervisor can assist by asking such questions as: "What did you 
think of die lesson?"; "Do you feel tiiat you met the goals you set out 
to accomplish?"; "What vwU you do differenfly in your reteach?**. If 

EVALUATION SHEET: SELF-ANALYSIS LESSON 



J 



Teacher Date 



Instructions: Complete questtcns 1 and 2 before your first lesson. 
I Complete questions 3 and 4 before your reteach lesson. 

! Complete question 5 after your reteach lesson. 



Take this completed form with you to the conference with 
your supervisor. 



1. List tho ob]ecth/es of your lesson. ^ ^ 



2. How do you intend to accomplish these objectives? 



3. How successful was the lesson? Cite positive and negative aspects. 



4. What changes do you intend to make for the reteach lesson? 



5. How effective were the changes you made from the first lesson? 
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the teacher identifies his own problems, he is much more likely to seek 
and reach solutions that will lead to improvement of that performance. 
Thirdly, if videotape replays are available, the supervisor should use 
this opportunity to reinforce effective behaviour, as well as to point out 
dcficiences. Whenever appropriate, the tape should be stopped, com- 
ments ^ven, and the segment replayed Eteferring comments untfl the 
end of the replay minimizes the reinforcement potential of the 
experience. 

Finally, evaluation forms provide another valuable source of 
feedback* They can be used to record reactions from students, super- 
visors, peers, and the teacher himself. A sample form that a teacher 
might use in analysing his own performance is reproduced m the 
accompanying figure. The question could obviously be reworked into 
reaction forms appropriate to students and supervisors. The most 
appropriate evaluation forms, however, are those designed to respond 
to the specific objectives of the microlesson and the individuals 
responsible for evaluation. 



CONCLUSION 

Microteaching should not be viewed as a panacea for all the problems 
related to the training of teachers. While it may, for example, help a 
teacher to become more conscious of his behaviour, it cannot transform 
a dull teacher into an intellectually exciting one. Its great advantage is 
that several valuable training functions occur simultaneously: it provides 
a setting that focuses on specific skills, enriches supervision, facilitates 
continuous training and research, and offers safe but realistic practice in 
instructional techniques. 

Above Jl, microteaching is a flexible teacher-training methcu and 
should not be abandoned if, for example, videotape equipment or 
experienced supervisors are unavailable. Since these and other variables 
can be adjusted, a precise definition of the microteaching context can 
vary according to the objectives desired and resources available. 
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THE IMPLEMENTATION OF WHO'S TRAINING 
PROGRAMME FOR TEACHERS OF 
THE HEALTH PROFESSIONS 



Tamds FiHop^ 



The background and content of WHO*s comprehensive, coordinated 
long-term programme for teachers of medicaJ and allied health sciences 
have been described elsewhere.^ This paper is concerned with the 
implementation of that programme, the purpose of which is to 
contribute to the improvement in the delivery of both preventive and 
curative health care. 

A Memorandum of Agreement was signed between WHO and the 
University of Illinois, Chicago, DL, USA, on 30 September 1970. It 
states in its preamble that **it is the desire and intention of the parties 
to bring about a comprehensive and co-ordinated training programme 
for teachers from schools of medicine and allied health sciences which 
will enable the participating WHO Re^ons to set up regjlonal and local 
(intercountry and country) Health Personnel Teacher Traming Centres 
and thereby contributing to improve not only the teaching standards 
of schools of health personnel, but also to meet the quantitative needs, 
i.e., manpower planning in relation to teaching staff". The operational 
paragraphs state, inter o/ifl, that: 

(1) The University shall provide in each of the academic years 1970/71 
through 1973/74 within and by its Center for Educational Development various 
training programmes and courses for selected candidates from each of WHO*s 
participating Regions, 

(2) ... the training programmes and courses shall include the following: 

(n) A programme of one-year duration for two to four fellows each academic 
year which may lead to a degree in Medical (Health Personnel) Education; 

(b) One course (seminar/workshop) of four weeks duration each academic 
year for 12 to 14 participants per course from WHO's participating Rc^ons; 



X Director, Divialoa of Hea!th Manpower Development, World Health Organization, Genera, 
Switzerland. 

• FQldp, T. (1973) Trabting teachtrs of health penonnel In: Development of edueatlonol 
Progrgmmes for the health professions, GencYa, Worid Health Organization {Public Health 
Papers, No. 52), p. 84. 



(c) One two to four week workshop on*site in the Regions each academic 
year for a maximum of 25 participants. 



The one-year course is designed to train the future full-time members 
of Regional Teacher Training Centres (RTTCs), among them the 
directors and deputy-directors. The 4-week seminar/workshops are 
intended for those who will become part-time teachers for the RTTCs. 
Finally, the 2-week on-site workshops are designed to "prepare the 
ground" for the RTTCs by discussing relevant problems with the 
teaching staff of the institution where the RTTCs will be located 

The goals and objectives for these three types of training have been 
properly defined (see Annex). The training activities within the frame- 
work of this Agreement started in June 1971, when the Center for 
Educational Development (CED), University of Illinois College of 
Medicine, Chicago, started to function as an Interre^onal Teacher 
Training Centre for Schools of Health Personnel (IRTTQ. 

In 1971 the WHO Regional Offices for Africa, the Eastern Mediter- 
ranean, South-East Asia, and the Western Pacific proceeded to select 
sites for RTTCs. First the Eastern Mediterranean, then the Western 
Pacific, and finally the South-East Asia regional offices called upon 
consultants to assist in the selection of a site and in drawing up a plan 
of operations. The site of the RTTC for the African Region was 
selected by the regional office staff. 

Consultant visits were arranged in Shiraz (Iran), Sydney (Australia), 
Bangkok (Thailand), and Peradeniya (Sri Lanka). For the purpose of 
these feasibility studies, criteria were elaborated and adopted by the 
consultant groups to standardize procedures for site selection. These 
criteria were intended to be used as a guide in the data collection and 
subsequent evaluation during consultants' visits. The consultants also 
drew up recommendations for the case where the site of an RTTC is 
selected by the regional office concerned. 

In 1971 - 1973 the regional offices proceded with the implementation 
of the programmes as follows: 

The WHO Regional Office for Africa assisted in the establishment of 
an English-language RTTC at Makerere University, Kampala (Uganda) 
and of a French-language RTTC at the University Centre of Health 
Sciences, Yaounde (Cameroon). 

The WHO Regional Office for the Americas assisted in the establish- 
ment of a Latin American Centre for Educational Technology in 
Health Sciences in Mexico City (Mexico) and another in Rio de Janeiro 
(Brazil), which perform the functions of an RTTC. 

The WHO Regional Office for the Eastern Mediterranean assisted 



90 

ERiC 



90 



in the establisment of an RTTC at the Pahlavi University Medical 
School, Shiraz (Iran). 

The Regional Office for Europe, considering teacher training to be a 
high-priority matter, planned activities such as educational meetings. 

The WHO Regional Office for South-East Asia assisted in the 
establisment of regional centres at the Faculty of Medicine, Chula- 
longkom University, Bangkok (Thailand) and at the Faculty of 
Medicine, University of Sri Lanka, Peradeniya (Sri Lanka). 

The WHO Regional Office for the Western Pacific assisted in the 
establishment of an RTTC at the University of New South Wales 
Medical Faculty, Sydney (Australia). 

The establishment of National Teachers Training Centres (NTTCs) is 
DOW being considered in all the re^ons. 

The following training activities have been conducted by the IRTTC 
within the scope of the above-mentioned agreement, with a view to 
preparing future leaders and collaborators of RTCCs, where leaders 
and teachers for NTTCs will later be prepared: three 1-year courses in 
1971/72, 1972/73, and 1973/74, with a total of 15 partidpants; two 
4-week seminar/workshop, in 1972 and one in 1973, with a total of 49 
participants; and a number of 2-week on-site workshops, including 
those in Sydney, Shiraz, and Bangkok (on-site workshops were also 
organized in many other places without the assistance of the IRTTC). 

An Interregional Seminar on Teacher Training for Schools of 
Medicine and Allied Health Sciences was held at the IRTTC in 1971, 
with 12 participants, including top-level administrators of schools for 
health personnel where RTTCs or NTTCs may be established. The 
purpose of this seminar was to ^ve partidpants a greater insight into 
the nature of educational leadership and the requirements for 
facilitating the implementation both of basic educational programmes 
for the health professions and of progranunes designed to train 
teachers for the health professions. A similar seminar in the French 
language was held in Yaounde (Cameroon) in 1972. 

The network of RTTCs in those WHO re^ons wishir-' to establish 
them was completed by the end of 1973. The first NTTCs started their 
activities in 1974, and all Member States wishing to establish such 
centres may have at least one by the end of this decade. Centres for 
educational research and development at the level of individual schools 
have already started to develop in several countries.^ A more rapid 
development may be expected in the second half of this decade after 
the NTTCs have started working. The activity of those centres in the 



^ See Educational rtseareh and development centres for tht health professions (tmpublldied 
document WHO/EDUC/74.172). 
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schools in the field of training, research, and service is, of course, not 
an end in itself. Their main purpose is to assist the schools in training 
health workers who will meet the real health needs and demands of 
the community they \wll later serve. 

Now, a further development is being considered — ^teacher training 
centres to serve as nuclei for the establishment of so-called health 
manpower development centres (HMDCs).^ These would be structures 
within the framework of which the three main elements of the health 
manpower delivery process — the planning, "production", and sur- 
veillance of health personnel — ^would be integrated. These three mam 
elements are mostly being developed separately in a somewhat isolated 
way, and their development has reached different stages in different 
countries. The planning of health manpower, i.e., defining the quality 
and quantity of manpower needed for the efficient coverage by health 
services of the entire population, is being developed in many countries, 
but in most cases it has no or hardly any influence on the second 
element, the "production" of manpower. This means that health man- 
power production is all too often isolated from the planning process 
and thus from the health services for which it is supposed to cater, 
with the result that the manpower that is trained is not always 
appropriate to the country's needs. The thud element, the monitoring 
of health manpower, i.e., the control of whether or not the tramed 
health personnel are able and ready to cope with the tasks for which 
they were trained and to feed the data back into the planning and 
production phases so that any necessary conections may be made (to 
close the cybernetic cycle of the manpower process), is hard to find 
anywhere. 

The mission of the planned HMDCs would be to promote the 
balanced development of these elements, to ensure that the health 
manpower process fulfils its main task, namely the provision and 
maintenance of manpower for the total health coverage of the entire 
population by health personnel who are motivated and able to serve 
their community. 



^ Such centres do not ecist anywhere yet, as they are only now (1974) being planned. 
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Annex 



GOALS AND OBJECTIVES FOR EDUCATIONAL ACTIVITIES 
TO BE CARRIED OUT BY 
THE CENTER FOR EDUCATIONAL DEVELOPMENT, 
UNIVERSITY OF ILLINOIS COLLEGE OF MEDICINE, 
CHICAGO, ILLINOIS, 
WITHIN THE FRAMEWORK 
OF THE WHO TEACHER TRAINING PROGRAMME 

ONE-YEAR FELLOWSHIP PROGRAMME 

A fellowship year at the Center for Educational Development provides 
a setting for an individual to define his role as an educational leader in the 
health professions. Some may elect to move towards this goal by enrolling 
in the Graduate Programme leading to a Master's degree in education, 
others by engaging in more focused work on a specific projec't. Whether 
through formal courses, independent study, or project work, each Fellow 
will be provided with the opportunity to develop skills in such areas as 
educational planning, educational programme implementation, and educat- 
ional research and evaluation. 

Educational planning 

Fellows will be afforded opportunities to engage in several kinds of 
educational planning during the course of a year, including short units 
of instruction (e.g., 1-3 hours), short-term training programmes (e.g., 1-4 
weeks) for health personnel, or instructional pro'^^^^mmes in the biomedical 
discipline from which they come. Participants i. , expect to acquire such 
skills essential for effective planning as: 

— deriving objectives from multiple sources 

— defining programme goals 

— formulating objectives in terms of expected student outcomes 

— using generally accepted principles of adult learning to develop strategies 
for implementing planned programmes 

— specifying concrete, manageable evaluation strategies 

articulating alternative techniques to achieving specified objectives, 
emphasizing the advantages and disadvantages of each 

— facilitating the active involvement of educational specialists when 
appropriate 

— identifying and utilizing varied kinds of educational resources 

— identifying potential sources of funding and procedures for acquiring 
them. 

Programme implementation 

In addition to increasing their competence in programme planning, 
participants will also be afforded multiple opportunities to acquire some 
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of the skUIs necessary for effective programme implementation. During the 
year they can expect to increase their competence in some or all of the 
following areas: 

— translating learning objectives into realistic instructional sequences 

— stimulating faculty and student enthusiasm and capturing commitment 
to a programme plan 

— facilitating the constructive resolution of interpersonal conflicts that 
interfere with the effective implementation of planned programmes 

— recognizing the advantages and disadvantages, as well as skilful use, of 
such methods as the lecture or small-group discussion 

— recognizing the strengths and limitations of such instructional media as 
films, audiotapes, television, and skilful incorporation of educational 
technology into instructional efforts 

— facilitating the professional growth of colleagues as teachers 

— assessment of personal as well as programme strengths and weaknesses 

— systematic experimentation with both organizational and personal imple- 
mentation strategies. 

Research and evaluation 

The focus of the fellowship programme will be upon the acquisition of 
those research and evaluation skills that can aid in the conduct of action 
research rather than in basic investigations. While it is unrealistic to 
expect that participants will become sophisticated investigators, the skills 
they acquire should make it possible for them to assess the major elements 
of student achievement and programme effectiveness. This capability should 
be manifested as increased proficiency in some of the following areas: 

— utilizing formative and summative evaluation procedures 

— selecting evaluation techniques that will measure at different levels: 

(a) cognitive processes 

(b) attitudes, interests, motivations 

(c) skills 

— constructing performance tests appropriate to such objectives 

— using and interpreting standardized evaluation instruments 

— interpreting evaluation data 

— formulating a research problem 

— utilizing work reported in the literature to identify areas for further 
study in medical education 

— using basic statistical techniques (e.g., probability, parametric and non- 
parametric statistics) 

— writing reports for publication. 



FOUR-WEEK seminar/workshop TEACHER TRAINING PROGRAMMES 

The gener^ purpose of the 4-week workshop is to provide participants 
with an opportunity to refine educational leadership skills that can be 
applied to the educational problems encountered in the home setting. 
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The intention is not that they will leave merely with an increased store of 
information about the educational process but rather that they will be 
better prepared to use both old and newly acquired knowledge in a manner 
that will: 

— stimulate local improvements in educational practices 

— provide a personal example of the application of educational principles 
to teaching activities 

— allow them to teach colleagues some of what they have learned. 
More specifically, attention will be directed towards five broad areas 

that represent the core of effective educational planning and implementa- 
tion: 

(1) The nature of learning 

— principles of adult learning 

— application to curriculum and instructional planning 

— experimental set 

(2) Curriculum development 

— the nature, source, and specification of educational objectives 

— influence of social forces upon programme planning 

— rationale for specific planning system 

— design of programme 

(3) The nature of teaching 

— selection of instructional methods 

— development of greater skill in selected techniques (e.g., lecture, 
group discussion) 

— sensitivity to individual and group needs 

(4) Instructional media 

— strengths and weaknesses of basic communication devices (e.g., 
slides, films, television, audiotapes) 

— selection of media 

— skill in preparation of simple media (e.g., 2 mm X 2 mm slides, 
audiotapes, transparencies) 

— more skilful use of selected media 

(5) Evaluation 

— principlt^s of student, teacher, or programme evaluation 

— strengths and weaknesses of specific testing methods (e.g., oral, 
essay, multiple-choice, practical examination) 

— assessment of problem-solving, clinical performance, attitudes 

— test scoring and reporting 

— use of tests as teaching tools. 

The programme is organized so as to allow each participant an 
opportunity to pursue in greater detail a special interest in one of these 
areas and will require all participants to develop at least one instructional 
unit that he can take lack to his home setting and share with colleagues. 



95 




9d 



The 4 weeks will include; 

— 3 sessions dealing with learning 

— 10 sessions dealing with curriculum (of which 2 will specifically relate 
to objectives) 

— 18 sessions dealing with teaching (including 5 half-days specifically 
devoted to microteaching, during which each participant will have an 
opportunity to teach his special instructional unit to others) 

— 10 sessions dealing with instructional media 

— 10 sessions dealing with evaluation principles and techniques. 

In addition, time will be set aside for individual reading, personal con- 
sultation with staff members, and participation in colloquia at which 
distinguished visitors will serve as discussion leaders. 



WHO ON-SITE WORKSHOPS 

General goal 

At the conclusion of a 2-wcek on-site programme each participant 
should be able to demonstrate that he comprehends the elementary com* 
poncnts of a sound educational process, that he can plan the application 
of such general principles to solve specific educational problems in his 
own setting, that he can use at least one instructional or evaluation 
technique with greater skill than prior to the workshop, and that h^^ 
appreciates the importance of systematic educational studies. 

Specific objectives 

Achievement of the general goals will be approached through the 
following activities in three broad areas: 

(1) Educational programme planning 

— identify major sr^rces of programme objectives, from within an 
institution and from outside 

— plan a rational system for the review and selection of institutional 
and departmental goals 

— identify conflicts in determining goals and priorities and suggest 
mechanisms for their resolution 

— recognize ways in which one's own actions may facilitate or impede 
the planning process 

— recognize the implications for teacher performance of a faculty 
commitment to independent student study. 

(2) Instructional strategies 

— identify the major elements of adult learning 

— recognize teaching practices that facilitate and impede learning 

— distinguish among cognitive, psychomotor, and affective learning 
objectives 

— write behavioural objectives in each of these domains 
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